BANNED USER: temperamental dysregulation
DaseindustriesLtd
late version of a small language model
Tell me about it.
User ID: 745
Banned by: @Amadan
DaseindustriesLtd
late version of a small language model
3yr ago
I've seen your comment btw. I agree that this is a thing, but it's disingenuous to act like such narrow compartmentalized object-level admissions have impact on the broader research program. Social scientists will often nod to «genes affect X» and go back to studying the effect of algorithmic bias on brain development. And of course Murray's book is only endorsed by other psychometrists; social scientists will at most recognize through gritted teeth they don't really have rebuttals.
UK may be more HBD-friendly than the US. You have Biobank. Americans have bona fide prohibition on relevant research.
DaseindustriesLtd
late version of a small language model
3yr ago
Can you clarify your reasons for joining the doomer camp? To be honest, I've been immensely disappointed by both sides of the debate, it feels like only randos on Twitter are clear-headed about it. Hinton with the *shocked* realization that his algorithms do work better than their biological counterparts, ridiculous naysayers who rely on shallow zingers and psychologizing, policy suits squawking over their sinecures, and the confused, exploited mob in the middle.
I think my perspective is clear enough. I don't care about stuff like muh spear-phishing and faked voices because it's just noise. I don't buy technical doom narratives out of Yuddist camp because they're bad Sci-Fi and crumble under scrutiny. I don't feel fear from the technically semi-literate ones («optimality», «instrumental convergence» stuff) because dangerously diverging designs yet capability-preserving designs are speculative and don't really make economic sense even at early steps. I don't worry about hacking, bioweapons and other serious problems based on amplification of human malice because they ought to be trivially surmountable with tool AIs of comparable level. In general, all AI doom plots just keep reinventing Bostrom's idea of a Singleton arising in a technically primitive world which is, of course, defenseless against it, and this isn't how this is playing out so far.
But above all, I have faith in the human will to power. We are not economic agents, we are apes. We rise to the top driven by the desire to see tiny apes below. This is both a blessing and a curse: the Tech Lords (or the government that expropriates their genies) are not willing to cede power to a glorified Microsoft Clippy, no matter how much better it gets at doing their own jobs. It's a curse because the top apes may not see much point, long-term, to preserving the proles in their current numbers and standing. But handing these elites the power to regulate proles out of this technology doesn't solve that issue! Distributing it widely does! Indeed, even the politicians in this hearing are appreciative of the empowerment effect that AI can provide, or at least pay lip service to it.
Do you just mean that GPT-5 would give OAI/MSFT too much of an edge? Or do you mean this level of capability in principle?
DaseindustriesLtd
late version of a small language model
3yr ago
Even very trivial common-sensical stuff like drone warfare and digital comms was being scoffed at, years in advance; I know of decent projects in the realm of electronic warfare that were turned down. Geeks would at least have established new opportunities for stealing, with some workable equipment trickling down to the force.
But the guys in charge had it all sorted out.
DaseindustriesLtd
late version of a small language model
3yr ago
Is this parody?
The future of AI is likely decided this week with Sam Altman's Congressional testimony. What do you expect?
Also testifying Tuesday will be Christina Montgomery, IBM’s vice president and chief privacy and trust officer, as well as Gary Marcus, a former New York University professor and a self-described critic of AI “hype.”
EDIT: the recording is here.
Frankly I've tried to do my inadequate part to steer this juggernaut and don't have the energy for an effortpost (and we're having a bit too many of AI ones recently), so just a few remarks:
-
AI Doom narrative keeps inceasing in intensity, in zero relation to any worrying change in AI «capabilities» (indeed, with things like Claude-100K Context and StarCoder we're steadily progressing towards more useful coding and paperwork assistants at the moment, and not doing much in way of AGI; recent results seem to be negative for the LLM shoggoth/summoned demon hypothesis, which is now being hysterically peddled by e.g. these guys). Not only does Yud appear on popular podcasts and Connor Leahy turns up on MSM, but there's an extremely, conspicuously bad and inarticulate effort by big tech to defend their case. E.g. Microsoft's economist proposes we wait for meaningful harm before deciding on regulations – this is actually very sensible if we treat AI as an ordinary technology exacerbating some extant harms and bringing some benefits, but it's an insane thing to say when the public's imagination has been captured by Yuddist story of deceptive genie, and «meaningful harm» translates to eschatological imagery. Yann LeCun is being obnoxious and seemingly ignorant of the way the wind blows, though he's beginning to see. In all seriousness, top companies had to have prepared PR teams for this scenario.
-
Anglo-American regulatory regime will probably be more lax than that in China or the Regulatory Superpower (Europeans are, as always, the worst with regard to enterpreneural freedom), but I fear it'll mandate adherence to some onerous checklist like this one (consider this as an extraordinary case of manufacturing consensus – some literally who's «AI policy» guys come up with possible measures, a tiny subset of the queried people, also in the same until-very-recently irrelevant line of work, responds and validates them all; bam, we can say «experts are unanimous»). Same logic as with diversity requirements for Oscars – big corporations will manage it, small players won't; sliding into an indirect «compute governance» regime will be easy after that. On the other hand, MSNBC gives an anti-incumbent spin; but I don't think the regulators will interpret it this way. And direct control of AGI by USG appointees is an even worse scenario.
-
The USG plays favourites; on the White House meeting where Kamala Harris entered her role of AI Czar, Meta representatives weren't invited, but Anthropic's ones were. Why? How has the safety-oriented Anthropic merited their place among the leading labs, especially in a way that the government can appreciate? I assume the same ceaseless lobbying and coordinating effort that's evident in the FHI pause letter and EU's inane regulations is also active here.
-
Marcus is an unfathomable figure to me, and an additional cause to suspect foul play. He's unsinkable. To those who've followed the scene at all (more so to Gwern) it is clear that he's an irrelevant impostor – constantly wrong, ridiculously unapologetic, and without a single technical or conceptual result in decades; his greatest AI achievement was selling his fruitless startup to Uber, which presumably worked only because of his already-established reputation as an «expert». Look at him boast: «well-known for his challenges to contemporary AI, anticipating many of the current limitations decades in advance». He's a small man with a big sensitive ego, and I think his ego will be used to perform a convincing grilling of the evil gay billionaire tech bro Altman. Americans love pro wrestling, after all.
-
Americans also love to do good business. Doomers are, in a sense, living on borrowed time. Bitter academics like Marcus, spiteful artists, scared old people, Yuddites – those are all nothing before the ever-growing legion of normies using GPT-4 to make themselves more productive. Even Congress staff got to play with ChatGPT before deliberating on this matter. Perhaps this helped them see the difference between AI and demons or nuclear weapons. One can hope.
Scott has published a minor note on Paul Ehrlich the other day. Ehrlich is one of the most evil men alive, in my opinion; certainly one of those who are despised far too little, indeed he remains a respectable «expert». He was a doomer of his age, and an advocate for psyops and top-down restrictions of people's capabilities; and Yud is such a doomer of our era, and his acolytes are even more extreme in their advocacy. Both have extracted an inordinate amount of social capital from their doomerism, and received no backlash. I hope the newest crop doesn't get so far with promoting their policies.
DaseindustriesLtd
late version of a small language model
3yr ago
According to my friend more involved with that scene, most Russian Sci-Fi and Fantasy authors are stalwart Z patriots now.
I think it's time we discussed the fiction writer question. It seems, unsurprisingly enough, that these people, left or right, are prone to believe in extreme, exaggerated political philosophies; and their genre fiction holds sway over nerds with a lack of common sense or taste and an excess of intellect, over people who can in fact influence the real world. Ukrainians ought to be thankful that the Russian state despised such people and their attempts at revolutionizing the military-industrial complex, so they merely coped, reading even more fiction (sometimes about popadantsy). In the west, however, Yud alone caused massive damage to our AI discourse. Gibson informed a bunch of negative progressive intuitions about information-age capitalism; Stephenson's naive libertarianism molded Silicon Valley and perhaps made the regulatory backlash inevitable. What are other examples?
DaseindustriesLtd
late version of a small language model
3yr ago
I do think that a rationally acting misaligned superintelligent AI, which is not a nonsense concept, will not see a reason to engage in spiteful behavior a la Basilisk, and also that the specific sort of irrationality that would make it spiteful is highly improbable to emerge as a result of AI research. But it's not logically absurd, in the way that timeless decision theory powering the vanilla Basilisk is; and if it were for some reason interested in that sort of stuff, I think it'd have been able to torture humans for at least centuries. It'd probably also be able to upload humans or construct random simulations and torture them, for whatever reason.
DaseindustriesLtd
late version of a small language model
3yr ago
Condescension is merited if you're asking for it; I mentioned IHNMAIMS in my post and this scenario is materially different from «you might be tortured and killed by a crazy person» in that it posits biological immortality.
I happen to think that a superintelligent being will at the very least be able to greatly prolong the life of its victim, this doesn't conflict with any part of our understanding of «the matter at hand» (we're pretty sure that a healthy brain can outlive the body if you provide it a decent environment). And of course this conjecture, while unproven, is vastly less far-fetched than continuity of identity between a human and a simulacrum built from indirect evidence.
DaseindustriesLtd
late version of a small language model
3yr ago
I really recommend reading the «I have no mouth and I must scream», or at least the synopsis.
DaseindustriesLtd
late version of a small language model
3yr ago
Not anthropomorphizing at all. This is straight up spiteful behavior, irrational waste of resources. As I've explained, the Basilisk AI has no incentive to do good on the threat, and thus cannot credibly precommit to it. It certainly benefits from us being convinced, but it's easier to have people believe something than to prove it true. In short, this is pretty uncertain because AIs needn't be fully rational decision makers (and indeed, human-mimics will not be), but I don't buy that timeless/logical decision theory is some sort of big brain invention that minds converge to.
Well, it adds the important dimension of unfriendly AI being spiteful and making a friend-enemy distinction. Actually, let's just scratch the whole alignment part and say Conjecture or Anthropic announce that, should they build a God-like AI, they'll go after people who tried to get in their way and torture them, while rewarding allies with extra share of the light cone. If they were very close to success, this would've been straightforwardly persuasive.
@Quantumfreakonomics seems to imply a much simpler and shorter -term Basilisk, like a misaligned GPT-5 model (or an aligned one from Anthropic) that literally sends robots to torture you, in the flesh.
It's a variant of I have no mouth and I must scream scenario, and I would argue it's at least plausible. It's not very different from normal political dynamics where the revolutionary regime persecutes past conservatives; and our theory of mind allows to anticipate this, and drives some people to proactively preach revolutionary ideals, which in turn increases the odds of their implementation. You don't really need any acausal trade or timeless decision theory assumptions for this to work, only historical evidence. As is often the case, lesswrongers have reinvented very mundane politics while fiddling with sci-fi fetishes.
Now one big reason for this not to happen is that a sufficiently powerful AI, once it's implemented, no longer cares about your incentives and isn't playing an iterative game. It loses nothing on skipping the retribution step. Unlike the nascent regime, it also presumably doesn't have much to fear from malcontents.
But assumption of perfect inhuman rationality is also a big one.
DaseindustriesLtd
late version of a small language model
3yr ago
…in conclusion, I think I'm starting to understand another layer of Krylov's genius. He had this recurring theme in his fictional work, which I considered completely meta-humorous, that The Powers That Be inject particular notions into popular science fiction, to guide the development of civilization towards tyranny. Complete self-serving nonsense, right? But here we have a regular sci-fi fan donning the mantle of AI Safety Expert and forcing absolutely unoriginal, age-old sci-fi/jorno FUD into the mainstream, once technology does in fact get close to the promised capability and proves benign. Grey goo (to divest from actually promising nanotech), AI (to incite the insane mob to attempt a Butlerian Jihad, and have regulators intervene, crippling decentralized developments). Everything's been prepped in advance, starting with Samuel Butler himself.
Feels like watching Ronnie O'Sullivan in his prime.
DaseindustriesLtd
late version of a small language model
3yr ago
Decent post with an overview of Yud's predictions: On Deference and Yudkowsky's AI Risk Estimates.
In general Yud was always confident, believing himself to know General High-Level Reasons for things to go wrong if not for intervention in the direction he advises, but his nontrivial ideas were erroneous, and his correct ideas were trivial in that many people in the know thought the same, but they're not niche nerd celebrities. E.g. Legg in 2009:
My guess is that sometime in the next 10 years developments in deep belief networks, temporal graphical models, … etc. will produce sufficiently powerful hierarchical temporal generative models to essentially fill the role of cortex within an AGI.… my mode is about 2025… 90% credibility region … 2018 to 2036
Hanson was sorta-correct about data, compute and human imitation.
But Yud has clout; so people praise him for Big Picture Takes and hail him as a Genius Visionary.
Excerpts:
At least up until 1999, admittedly when he was still only about 20 years old, Yudkowsky argued that transformative nanotechnology would probably emerge suddenly and soon (“no later than 2010”) and result in human extinction by default. My understanding is that this viewpoint was a substantial part of the justification for founding the institute that would become MIRI; the institute was initially focused on building AGI, since developing aligned superintelligence quickly enough was understood to be the only way to manage nanotech risk…
I should, once again, emphasize that Yudkowsky was around twenty when he did the final updates on this essay. In that sense, it might be unfair to bring this very old example up.
Nonetheless, I do think this case can be treated as informative, since: the belief was so analogous to his current belief about AI (a high outlier credence in near-term doom from an emerging technology), since he had thought a lot about the subject and was already highly engaged in the relevant intellectual community, since it's not clear when he dropped the belief, and since twenty isn't (in my view) actually all that young.
In 2001, and possibly later, Yudkowsky apparently believed that his small team would be able to develop a “final stage AI” that would “reach transhumanity sometime between 2005 and 2020, probably around 2008 or 2010.”
In the first half of the 2000s, he produced a fair amount of technical and conceptual work related to this goal. It hasn't ultimately had much clear usefulness for AI development, and, partly on the basis, my impression is that it has not held up well - but that he was very confident in the value of this work at the time.
The key points here are that:
- Yudkowsky has previously held short AI timeline views that turned out to be wrong
- Yudkowsky has previously held really confident inside views about the path to AGI that (at least seemingly) turned out to be wrong
- More generally, Yudkowsky may have a track record of overestimating or overstating the quality of his insights into AI
Although I haven’t evaluated the work, my impression is that Yudkowsky was a key part of a Singularity Institute effort to develop a new programming language to use to create “seed AI.” He (or whoever was writing the description of the project) seems to have been substantially overconfident about its usefulness. From the section of the documentation titled “Foreword: Earth Needs Flare” (2001):
…Flare was created under the auspices of the Singularity Institute for Artificial Intelligence, an organization created with the mission of building a computer program far before its time - a true Artificial Intelligence. Flare, the programming language they asked for to help achieve that goal, is not that far out of time, but it's still a special language.”*
A later piece of work which I also haven’t properly read is “Levels of Organization in General Intelligence.” At least by 2005, going off of Yudkowsky’s post “So You Want to be a Seed AI Programmer,” it seems like he thought a variation of the framework in this paper would make it possible for a very small team at the Singularity Institute to create AGI
In his 2008 "FOOM debate" with Robin Hanson, Yudkowsky confidentally staked out very extreme positions about what future AI progress would look like - without (in my view) offering strong justifications. The past decade of AI progress has also provided further evidence against the correctness of his core predictions.
When we try to visualize how all this is likely to go down, we tend to visualize a scenario that someone else once termed “a brain in a box in a basement.” I love that phrase, so I stole it. In other words, we tend to visualize that there’s this AI programming team, a lot like the sort of wannabe AI programming teams you see nowadays, trying to create artificial general intelligence, like the artificial general intelligence projects you see nowadays. They manage to acquire some new deep insights which, combined with published insights in the general scientific community, let them go down into their basement and work on it for a while and create an AI which is smart enough to reprogram itself, and then you get an intelligence explosion…. (p. 436)
When pressed by his debate partner, regarding the magnitude of the technological jump he was forecasting, Yudkowsky suggested that economic output could at least plausibly rise by twenty orders-of-magnitude within not much more than a week - once the AI system has developed relevant nanotechnologies (pg. 400).[8]
I think it’s pretty clear that this viewpoint was heavily influenced by the reigning AI paradigm at the time, which was closer to traditional programming than machine learning. The emphasis on “coding” (as opposed to training) as the means of improvement, the assumption that large amounts of compute are unnecessary, etc. seem to follow from this. A large part of the debate was Yudkowsky arguing against Hanson, who thought that Yudkowsky was underrating the importance of compute and “content” (i.e. data) as drivers of AI progress. Although Hanson very clearly wasn’t envisioning something like deep learning either[9], his side of the argument seems to fit better with what AI progress has looked like over the past decade.
In my view, the pro-FOOM essays in the debate also just offered very weak justifications for thinking that a small number of insights could allow a small programming team, with a small amount of computing power, to abruptly jump the economic growth rate up by several orders of magnitude:
- It requires less than a gigabyte to store someone’s genetic information on a computer (p. 444).[11]
- The brain “just doesn’t look all that complicated” in comparison to human-made pieces of technology such as computer operating systems (p.444), on the basis of the principles that have been worked out by neuroscientists and cognitive scientists.
- There is a large gap between the accomplishments of humans and chimpanzees, which Yudkowsky attributes this to a small architectural improvement
- Although natural selection can be conceptualized as implementing a simple algorithm, it was nonetheless capable of creating the human mind
In the mid-2010s, some arguments for AI risk began to lean heavily on “coherence arguments” (i.e. arguments that draw implications from the von Neumann-Morgenstern utility theorem) to support the case for AI risk. See, for instance, this introduction to AI risk from 2016, by Yudkowsky, which places a coherence argument front and center as a foundation for the rest of the presentation.
However, later analysis has suggested that coherence arguments have either no or very limited implications for how we should expect future AI systems to behave. See Rohin Shah’s (I think correct) objection to the use of “coherence arguments” to support AI risk concerns. See also similar objections by Richard Ngo and Eric Drexler (Section 6.4).
Well you're not a true believer in Yuddism nor neurotic in the right way so that's pretty much expected.
And what do you get if you distill LLMs on human cognition and thoughts (the internet)? You get something that thinks remarkably like us.
Yes, this happens for understandable reasons and is an important point in Pope's attack piece:
The manifold of possible mind designs for powerful, near-future intelligences is surprisingly small. The manifold of learning processes that can build powerful minds in real world conditions is vastly smaller than that.…
The researchers behind such developments, by and large, were not trying to replicate the brain. They were just searching for learning processes that do well at language. It turns out that there aren't many such processes, and in this case, both evolution and human research converged to very similar solutions. And once you condition on a particular learning process and data distribution, there aren't that many more degrees of freedom in the resulting mind design. To illustrate:
1 Relative representations enable zero-shot latent space communication shows we can stitch together models produced by different training runs of the same (or even just similar) architectures / data distributions.
2 Low Dimensional Trajectory Hypothesis is True: DNNs Can Be Trained in Tiny Subspaces shows we can train an ImageNet classifier while training only 40 parameters out of an architecture that has nearly 30 million total parameters.
The manifold of mind designs is thus:
1 Vastly more compact than mind design space itself.
2 More similar to humans than you'd expect.
3 Less differentiated by learning process detail (architecture, optimizer, etc), as compared to data content, since learning processes are much simpler than data.
(Point 3 also implies that human minds are spread much more broadly in the manifold of future mind than you'd expect, since our training data / life experiences are actually pretty diverse, and most training processes for powerful AIs would draw much of their data from humans.)
etc. LLM cognition is overwhelmingly data-driven; LLM training is in a sense a clever way of compressing data. This is no doubt shocking for people who are wed to the notion of intelligence as an optimization process, and trivial for those who've long preached that compression is comprehension; but same formalisms describe both these frameworks; preferring one over the other is a matter of philosophical taste. Of course, intelligence is neither metaphor, by common use and common sense it's a separate abstraction; we map it to superficially simpler and more formalized domains, like we map the historical record of evolution to «hill-climbing algorithms» or say that some ideas are orthogonal. And it's important not to get lost in layers of abstraction, maps obscuring territory.
Accordingly I think and argue often that ANNs are unjustly maligned and indicate a much more naturally safe path to AGI than AI alignists' anxious clutching to «directly code morality and empathy into a symbolic GOFAI or w/e idk, stop those scary shoggoths asap». (With embarrassing wannabe Sheldon Cooper chuunibyou gestures for emphasis. Sorry, I'm like a broken record but I can't stop noticing just how unabashedly cringe and weirdly socialized these people are. It's one thing to act cute and hyperbolic in writing on a forum for fellow anime and webcomic nerds, very different to grimace in the company of an older person when answering about a serious issue. Just pure juvenility. Brings back some of my most painful elementary school memories. Sure I should cut him slack for being an American and Bay Aryan, but still, this feels like it should be frowned upon, for burning the commons of the dynamic range if nothing esle).
…But that's all noise. The real question is: how did Yud develop his notion of The Total Mind Space, as well as other similar things in the foundation of his model? It's a powerful intuition pump for him, and now for his followers. There's this effectively infinite space of Optimization Processes, and we «summon» instances from there by building AIs they come to possess. Surely this is just an evocative metaphor? Just a talented writer's favourite illustration, to break it down for normies, right? Right? I'm not sure that's right. I think he's obsessed with this image well beyond what can be justified by the facts of the domain, and it surreptitiously leaks into his reasoning.
In principle, there are infinitely many algorithms that can behave like a given LLM, but operate on arbitrarily alien principles. Those algorithms exist in that hypothetical Total Mind Space and we really cannot predict how they will act, what they really «optimize for»; the coincidence of their trajectory with that of an LLM (or another model) that earnestly compressed human utterances into a simple predictive model gives us no information as to how they'll behave out of distribution or if given more «capacity» somehow. Naturally this is the problem of induction. We can rest easy though: the weirder ones are so big they cannot possibly be specified by the model's parameters, and so weird they cannot be arrived at via training on available data. That is, if we're doing ML, and not really building avatars to channel eldritch demons and gods who are much greater than they let on.
I am not aware of any reason to believe he ever seriously wondered about these issues with his premises, in all his years of authoritatively dispensing AI wisdom and teaching people to think right. I covered another such image, the «evolution vs SGD», recently, and also the issue of RL, reward and mesa-optimization. All these errors are part of a coherent philosophical structure that has fuck all to do with AI or specifically machine learning.
See, my highest-order objection is that I dislike profanation. …not the word. In English this seems to have more religious overtones but I just mean betrayal of one's stated terminal principles in favor of their shallow, gimmicky, vulgar and small-mindedly convenient equivalent (between this and poshlost, why do we have such refined concepts for discussing cultural fraud?) Yud aspired to develop Methods of Rational Thinking but created The Way Of Aping A Genius Mad Scientist. Now, when they observe something unexpected in their paradigm – for example, « Godlike AI being earnestly discussed in the mainstream media» – they don't count this as a reason to update away from the paradigm, but do exactly the opposite, concluding that their AI worries are even truer than believed, since otherwise we wouldn't have ended up in a «low-probability timeline». It's literally a fucked-up episemology on par with worst superstitions; they've fashioned their uncertain beliefs into ratchets of fanaticism (yes, that's Kruel again).
This reveals a qualitatively greater error of judgement than any old object-level mistake or overconfidence about odds of building AI with one tool or another. This is a critical defect.
Yud is not trying to sway honest-to-God normies with this podcast tour (and people who've greenlit this multipronged astroturf of AI doom don't expect him to either, but that's a conspiratorial aside). He never could be popular among normies, he never will, he's smart enough to realize this. His immediate target is… well, basically, nerds (and midwitted pop-sci consumers who identify as nerds). Nerds in fact appreciate intellectual aggression and domination, assign negligible or negative weight to normie priors like «looks unhinged, might be a crackpot», and nowadays have decent economic power, indeed even political power in matters pertaining to AI progress. Nerds are not to be underestimated. When they get serious about something, they can keep at it for decades; even an autist entirely lacking in affective empathy and theory of mind can studiously derive effective arguments over much trial and error, and a rationalist can collate those tricks and organize an effective training/indoctrination environment. Nerds will get agitated and fanatical, harangue people close to them with AI doom concerns, people will fold in the face of such brazen and ostensibly well-informed confidence, and the future will become more aligned with preferences of doomers. Or so the thinking goes. I am describing the explicit logic that's become mainstream in one rat-adjacent community I monitor; they've gotten to the stage of debating pipelines for future AI-Alignment-related employment, so that the grift would never stop.
But on the object level:
Here's my advice: if you want to convince people who are not already steeped in your philosophy you need to have a short explanation of your thesis that you can rattle off in about 5 minutes that doesn't use any jargon the median congresscritter doesn't already know. You should workshop it on people who don't know who you are, don't know any math or computer programming and who haven't read the Sequences, and when the next podcast host asks you why AIs will kill us all, you should be able to give a tight, logical-ish argument that gets the conversation going in a way that an audience can find interesting.
You assume there is a minimal viable payload that he has delivered to you and others, and which is viable without all that largely counterproductive infrastructure. That is not clear to me. Indeed, I believe that the whole of Yud's argument is a single nonrobust just-so narrative he's condensed from science fiction in his dad's library, a fancy plot. It flows well, but it can be easily interrupted with many critical questions. He describes why timelines will «converge» on this plot, the nigh-inevitability of that convergence being the central argument for urgency of shutting down AI, but its own support is also made up of just-so stories and even explicit anti-empiricism; and once you go so deeply you see the opposite of a trustworthy model – basically just overconfident logorrhea.
That's exactly why Yud had to spend so many years building up the delivery vehicle, an entire Grand Theory, an epistemological-moral-political doctrine, and cultivating people who take its premises on faith, who all use the same metaphors, adhere to the same implicit protocol. His success to date rests entirely on that which you're telling him to drop.
Here's how Zvi Moskowitz understands the purpose of Yud's output, «its primary function is training data to use to produce an Inner Eliezer that has access to the core thing». (Anna Salomon at CFAR seems to understand and apply the same basic technique even more bluntly: «implanting an engine of desperation» within people who are being «debugged»).
In a sense, the esotericism of Yuddite doctrine is only useful, it had insulated people from pushback until they became rigid in their beliefs. Now, when you point at weak parts in the plotline, they answer with prefab plot twists or just stare blankly, instead of wondering whether they've been had.
Nerdy sects work; Marxism only the bloodiest testament to this fact. Doomsday narratives work too, for their target audience (by the way, consider the similarity of UK's Extinction Rebellion and Yuddites' new branding «Ainotkilleveryoneism»). They don't need to work by being directly compelling to the broader audience or by having anything to do with truth.
P.S. Recently I've encountered this interesting text from an exactly such an AI-risk-preoccupied nerd as I describe above: The No-Nonsense Guide to Winning At Social Skills As An Autistic Person
Pick a goal large enough to overcome the challenges involved.
Self-improvement is hard work, and that goes double whenever you’re targeting something inherently difficult for you (e.g. improving social skills as an autistic adult). This is the part where I most often see autistic adults fail in their efforts to improve social skills. Often, they pick some sort of goal, but it’s not really based in what they truly want. If your goal is “conform to expectations,” that goal is not large enough to overcome the challenges involved. If your goal is “have people feel more comfortable around me,” that goal is not large enough to overcome the challenges involved. If your goal is “stop a terrorist cell from destroying the Grand Coulee Dam, flooding multiple cities in Washington, and wiping out the power grid along much of the West Coast,” that goal is large enough..
However, not all of us are Tom Clancy protagonists, and so a typical goal will not end up being that theatrical in nature. Still, once you’ve found something genuinely important to do in your life, and you feel that improving your social skills will dramatically improve your ability to carry that out, this will tend to serve as a suitable motivation for improving your social skills. These will overwhelmingly tend to be altruistically motivated goals, as goals that are selfish in nature will tend to be less motivating when things get hard for you personally. For me, goals related to Effective Altruism serve that role quite well, but your mileage may vary.
His social well-being is now literally predicated on his investment in EA-AI stuff, so I'd imagine he goes far, and this easily counts more for Yud's cause than 10k positive comments under another podcast.
DaseindustriesLtd
late version of a small language model
3yr ago
I do not report you because I practically never do but I would like to ask you not to use basic slurs. I don't care about it but others may care and this poisons the site for neophytes. Let's not slide into /r/CultureWarRoundup even faster.
Checked three random songs with whisper.cpp (ggml-medium.bin -l en): Grimes' 4ÆM, Kanye's Champion, and Juno Reactor's Angels and Men. Kanye got parsed ≈perfectly, the other two gave me lines of [MUSIC] [MUSIC] [MUSIC] and (dramatic music) (dramatic music) (dramatic music), which is fair enough but very helpful. On this note, it often interprets silence at the end of my voice notes into nonsense like «subtitle editor Lena Yeterenko, corrector T. Ilyin» or «[popular streamer] is done for today, like & subscribe», which I vaingloriously attribute to my professional diction and sexy voice despite it really being just an archetypal case of overfitting. Nevertheless, I think with a little tinkering this is all doable with basic Whisper and that's probably what runs on the backend of your service. We're 99% of the way to solving transcription.
And a bunch of other tasks, to put it mildly. Ethan Mollick recently wrote a nice piece on ChatGPT+Code Interpreter: It is starting to get strange. (My somewhat-related earlier comment).
Code Interpreter is GPT-4 with three new capabilities: the AI can read files you upload (up to 100MB), it can let you download files, and it lets the AI run its own Python code. This may not seem like a huge advance, but, in practice, it is pretty stunning.
Lets take an example: I am writing a blog post about how amazing ChatGPT is at working with code right now. I would like you to create the perfect illustration, a GIF using Python, that represents this ability. Decide what an appropriate amazing GIF would be, then figure out how to create it and let me download it. After its first attempt, I encouraged it to do something even more creative. It decided on a strategy, wrote software to enact its strategy given the constraints on its tools, executed the code, and gave me a download link to a GIF.
…So the AI shows genuine creativity in problem solving. That seems like a big deal, but not actually the big deal I want to discuss. I want to show you that Code Interpreter has turned GPT into a first-rate data analyst. Not a data analysis tool, but a data analyst. It is capable of independently looking at a dataset, figuring out what is interesting, developing an analytical strategy, cleaning data, testing its strategy, adjusting to errors, and offering advice based on its results.
An example: I uploaded a Excel file, without providing any context, and asked three questions: "Can you do visualizations & descriptive analyses to help me understand the data? "Can you try regressions and look for patterns?" "Can you run regression diagnostics?" It did it all, interpreting the data and doing all of the work - a small sample of which is below.
etc. I'm sure that a great deal of blue-collar work of this kind, work that is well paid for, is actually not very valuable, if not outright bullshit in the style of that Bernanke&Krugman joke: routine extraction of so-what observations and weaving them into platitudinous narratives for non-actionable powerpoint presentations for people who make presentations of their own to drive up the company valuation by justifying the foundation of the whole house of cards.
But some of it is valuable – immensely so. Markets have not yet recognized that we have an alien artifact on our hands. It may create unemployment, albeit that's not a given; it definitely is able to add value comparable with the bigger breakthroughs of the 20th century. And this is just about GPT-4-tier systems.
Yud of course prophesied that AIs won't be very useful until very late in the game where they rapidly become deadly (I can't be bothered right now to hunt for specific quotes). I think it's in clear sight that even just continuation of this line of tools will be sufficient to accelerate basic research to the point that we get either controlled nanotechnology or proof of its unfeasibility before evil agentic AI pulls itself up by its tentacles and killseveryone. Not to mention the possibility to maintain decently efficient local supply chains and other infrastructures even for small off-grid communities, obviating major value propositions of a nation state.
But the more likely scenario, IMO, is that «evidence» from recent and upcoming Hollywood flicks, Yud's podcasts/TED talks and obsessive fearmongering of mentally unstable people will be used to justify severe «compute governance». The Elders seem to be of the mind that useful artificial intelligence is no toy for children. You've got your guns, you've got your trucks, you've got your free press and your flags; and be happy with those inconsequential obsolete freedoms. LARP as a Butlerian Jihadist (while swiping vigorously on your personal data harvester device) or WH40K's loyal citizen (temporarily embarrassed by an onslaught of Slaanesh's forces) if that's your flavor of cope – just do not aim above your station. No «digital god» for you, little man; pray to your paper gods.
DaseindustriesLtd
late version of a small language model
3yr ago
One of my stranger brainfarts. Never knew your embeddings are so similar.
what is it with Russian men and the most unflattering buzzcuts?
Thailand is too hot in this season for them to make the answer obvious with the complementary garb.
I have some acquaintances in Thailand, could ask later.
My take is, basically there are two things to it. One is a rather dumb post-Soviet prison-informed culture of masculinity that is summed up in the saying «a Man must be not much prettier than an ape», and the suspicion that anyone who tries to look better is an anus-bleaching sissy faggot prison bitch. Thankfully this is a passing sentiment (passing away together with heteronormativity, some would lament) but it still holds sway over older, provincial and lower-class Russians. You don't wear bright colors you don't style your hair you double don't use cosmetics and you go for the most utilitarian, anti-aesthetic, no fucks given look possible. Perplexingly, you also don't have to work out, so this is actually the easy life.
Plus short buzzcuts can be maintained on your own with a trimmer, and feel nice. Many acquire the habit in the army, I gather.
Another is that many among us age ungracefully. Probably mostly alcohol, smoking, climate and stress, but must be some irreducible genetic contribution too. Russians, particularly men, die early, develop chronic age-associated diseases early and start to go bald early, or so it seems. Though statistics point to Caucasians being hard-hit by alopecia in general, with many of our socioeconomic betters being worse off. Not sure if this accounts for age and dynamics. Anyway, buzzcut – with the above justification – is the natural coping strategy when you see hair on your pillow and Norwood in the mirror.
Summed up and polished with some soy sausages and phytoestrogen-heavy beer, those pressures, I surmise, produce the Skuf phenotype and what I call Skufization syndrome – the rapid onset physical deterioration that hits Russian men in their late 20s. It's on us to research this malady and develop countermeasures.
Relatedly, one of the things that surprised me in Turkey was massive numbers of men with freshly done hair transplants, their scalps peppered with red dots, back of their heads clad in bandages (is it supposed to look so crude?).
@orthoxerox may have a better idea.
Low-rigor response because I think you do have to study the object level a bit to evaluate the sides.
What precisely g is?
Precisely what its definition says. As Jensen himself put it:
It… reflects individual differences in performance on tests or tasks that involve any one or more of the kinds of processes just referred to as intelligence. The g factor emerges from the fact that measurements of all such processes in a representative sample of the general population are positively correlated with each other, although to varying degrees. A factor is a hypothetical source of individual differences measured as a component of variance. The g factor is the one source of variance common to performance on all cognitive tests, however diverse.
This is the definitive blog post refuting a popular methodological criticism.
Do you mean what it corresponds to in reality? I suppose it's just a holistic brain performance index, that's also predictive of general health. There isn't one physical thing that creates g, but the sum of diverse brain factors (half of our genome gets expressed in the brain) ensures that it emerges in the factor structure of our mental abilities, conditional on similar amount of training useful for each, and that predicts both general functioning and peak achievement. It isn't surprising that our abilities are highly correlated. We have a very homogenous brain made up of extremely complex computational elements (neurons) implementing simple task-agnostic learning algorithms, so most of the complexity and variation of our basic architecture (though not specific structure acquired over the lifetime) is shared between neurons or in their generic connectivity. Random genetic errors or (non-localized) environmental insults create different perturbations of the neuronal function, but on the whole-brain level they all push the system away from its optimal regime, no matter what it's trying to learn or to perform. You're unlikely to score in the 99th percentile on arithmetic and 20th on vocabulary if your axonal conduction is shot or your synapses are too sparse or do not get pruned well or your total cell count is too low or if your cell migration was too noisy or intracellular metabolism is somehow defective. And even if you've somehow developed specific tricks to cope very well with your shortcomings in a given skill, on the population level the power of the general factor becomes overwhelming.
Much of the issue is reducible to signal/noise ratio. High-performing brains ride the edge of metastability and wrangle representations easily; low-performing ones waste power fending off chaos and lose track of the context.
The concept of heritability and how it relates or doesn't relate to genetic causes of individual or group differences. I am aware of the "books at home" example. Is that all there is to it?
Steelman(!) Turkheimer's position. No, I don't want to hear about his politics.
But it is politics ultimately: we should be super, duper, ultra skeptical of HBD-related stuff and probably censor it for good measure, because Holocaust. He is leery of political implications of HBD acknowledgement regardless of specific merits of a given study.
Closer to the object level, I think he was recently steelmanned by Tailcalled:
People might read that intelligence is genetically correlated with myopia, or that homosexuality is genetically correlated with depression, and conclude that these are due to a direct biological link, rather than due to smarter people straining their eyes reading or staying indoors more, or homosexual people being discriminated against. Yet as we saw with education, this assumption is unwarranted; phenotypic causality leads to heritability.
Or put another way: the whole causal chain between genes and outcomes matters, for it may have links that depend on contingent properties of the environment. If proven, this would invalidate our assumption about this trait's heritability in a more general case. In our ancestral environment, smarter people probably hadn't been more myopic, because there had not existed the contingent segment «smart – reads a lot – sits indoors more than others – stupid monkeys haven't yet built good indoors lighting – not enough dopamine signaling in the retina – extended axial elongation period – myopia». It can be highly heritable, we can even find explanatory genetic polymorphisms – but the relationship disappears whenever we stop having our smartest children spend more time indoors with poor illumination. Likewise for all contingent chains. Therefore, even though everything is heritable, the first law of behavioral genetics does not mean that genetics is destiny or that people's phenotypes are molded by their genes in the intuitive strong sense the texture of their hair is.
This line of critique is pretty old, used by Lewontin etc. already, and is fair enough on its face, but assumes that some common mitigations psychometrists know about are insufficient, and it's only when it comes to proving this that we see clearly how Turkheimer is politically driven.
Roughly summarise the position of Kirkegaard et al.
Roughly:
We are living in a saturated and humane age. Human brain development is actually surprisingly robust and insensitive to inputs that are scarce in developed economies (even if some of them were scarce not so long ago in the past), and human cognitive development mostly proceeds close to the optimal way so long as you don't severely and obviously fuck up brain development or deviate far from low-effort common sense in nurture. Thus, developed societies have exhausted reasonable interventions that target contingent hazards disproportionately affecting g in different demographics; probably all hazards that bring down g in the population at large. While some differences in positive factors remain, they're long in the diminishing returns regime for more advantaged groups, as far as intelligence is concerned; and negative factors are similarly minor and maintained not by any iniquity, scarcity or coercion but out of their free choice by people who are worse off; and the rest is completely unsystematic. There's no low-hanging fruit left. No more toxic lead paint we can scrape off walls of homes where redlined minorities live, no malnutrition we can solve with food stamps, no miracle iodine-enriched brain-enhancing diet that only upper class kids are getting, no worms in ponds peasants have to drink from, no education reform that can remove some unnecessary cultural pressure. We've optimized and homogenized our environment to the point that practically all subpopulations realize the same, very high, percentage of their genotypic potential for g that is possible at the current technological/infrastructural level, and so differences in outcomes predicated on differences in g are explained by true differences in heredity, and are not amenable to elimination via any social policy change we'd recognize as fair.
Mostly the same logic applies to non-cognitive factors that influence outcomes, if to a lesser extent.
This model is corroborated by an extreme wealth of evidence and by the complete failure to create a competitively powerful model that rejects its premises, despite nearly a century of trying to do so, generous investment and genuine desire of talented researchers. This knowledge is being obfuscated and suppressed by political means.
DaseindustriesLtd
late version of a small language model
3yr ago
humans have a moral skeleton, innate hardwiring that allows us to learn morality
I suppose we have, to some extent, but it can't be all that robust. Tons of species are psychopathic by our standard, and of course this standard exists to distinguish humans who don't fit it. So it's more like a slight inductive bias, in the same way we have biases to learn to navigate in 3D space and prefer sugary foods. Biases of the algorithm can be substituted with biases in the data.
and believe it (as opposed to mimic it). This is highly instrumentally non-convergent and probably needs to be coded into an AI directly; gradient descent on output will only produce lying psychopaths mimicking morality.
I don't see why that would be true. Indeed, I do not see why gradient descent wouldn't be much better of learning deep cognitive regularities including morality. You seem to hold that morality is something essential, some set of terminal value-influences, but why is that true for morality and not any other aspect of our cognition, both instrumentally worthwhile and «instrumentally non-convergent» ones? Every part of our decision-making feels profoundly qualitatively colored for us.
Why is "coded directly" better than learned? The major reason we're doing this stuff is that it's vastly better at generalization,
Sorry, this looks like a vibe-based argument, where neural nets deceptively "imitate" and hard code is "good and honest". It's all algorithms. Inasmuch as human minds are computable, our morality is an algorithm too.
GOFAI has some hope because we could code morality directly
What good would that do? It'd break OOD just the same, and if it didn't break, it'd be rewritten or worked around by the purported daemon of optimization.
But I'm not seeing any hope of success on non-uploads without the ability to look inside the box. This is because "is moral" and "is pretending to be moral successfully" have identical output except in situations where dropping the pretence is worth it i.e. situations where there's a high chance of you losing control upon betrayal.
Reminder that LLMs cannot learn to do the «treacherous turn» because the dreaded SGD mercilessly ablates cognitive routines that do not contribute to decreasing loss in training. This, of course, holds in the general case.
But even beyond that, outputs may be similar but activations aren't, we know how to look at activations, and we know there are differences between the model subjectively evaluating its output as true or false.
it seems like the difficulty of determining whether spaghetti-code does X is generally at least as high as the difficulty of writing code that does X, which implies that making safe NNs is at least as hard as writing GOFAI
No, generation is always vastly simpler than classification unless you require classification that reconstructs the process of generation, of course.
I've long held that Yuddist program is, in addition to all else, an attractor for a particular variation of anxiety/OCD disorder: fetishization of «being in control», of making thoughts play by «proper rules». But it's fetishization because it doesn't really work, it pursues ghosts, precisely the deceptive external form of reliability. You gain clarity of ideas by testing them against evidence, not by being real suspicious of disconfirmations.
Pitts wrote that his depression might be “common to all people with an excessively logical education who work in applied mathematics: It is a kind of pessimism resulting from an inability to believe in what people call the Principle of Induction, or the principle of the Uniformity of Nature. Since one cannot prove, or even render probable a priori, that the sun should rise tomorrow, we cannot really believe it shall.”
This malady is to be treated, not nurtured.
DaseindustriesLtd
late version of a small language model
3yr ago
Do you believe that humans must be utilitarians to achieve success in some task, " in the sense that there must exist a function V(s) that takes in possible world-states s and spits out a scalar, and the human's behaviour can be modelled as maximising the expected future value of V(s)"?
DaseindustriesLtd
late version of a small language model
3yr ago
i don't believe cows or chickens possess a meaningful capacity to suffer. pigs probably suffer more
How does this work on the substrate level? You may like pigs more or whatever but they're clearly organisms on the same level of sophistication as cows. (Naturally humans are not far off from either)
Like what?
I'm not American, I don't owe it to your paranoid star-sprangled hivemind to pretend that China is a thing worth paying attention to. There is no «Yellow menace», there is no «threat of Chinese eugenics»; for the world at large, China is about as relevant as Czech Republic, only quantitatively bigger. Do you want to talk about the Czech AI threat? If you want to talk about China, we can go off vibes. My read on vibes is diametrically opposite to yours. If you want to discuss the evidence, well, what is the evidence for this purported Chinese focus on AI?
Because the CCP is well-known for cutting uppity businessmen down to size, and AI to the party bosses looks like «blockchain» or «fintech» – some new grifting scheme to syphon off some of their control over the system; another invention that's a bigger internal threat than external competitive edge. Remember when Americans were afraid that Choyna, ever ruthless and game-theoretically diabolical, will leverage their dominance in cryptocurrency mining? They've gladly regulated it out of existence instead.
Yeah, it's accepted by Americans, but does China notice that they're in an AI race? For all I know they're of the mind that semiconductors are needed only for drone warfare over the first island chain, monitoring Uighur camps and manufacturing automation – or, perhaps, to produce high-end smartphones; which is why the severity of sanctions and impossibility of compromise befuddles them so. Many Americans are, indeed, obsessed with geopolitical dominance of their Empire of Freedom, like some Avengers franchise characters or, less charitably, suicidal ants willing to lay down their livelihoods for the largesse of the colony. But I don't notice the same spirit in Chinese people; they're selfish, entrepreneurial, too engrossed with busywork to notice the big picture. Does Baidu or ByteDance believe they're forging the future of the lightcone? What are the names of Chinese Hassabis or Altman? How many mainlanders are even aware of this eschatological discourse?
Ironically, the underlying model was Stable Diffusion, or specifically a minor finetune of NovelAI. Stability is incorporated in London, UK. Novel – Delaware, US. Alibaba has never released Composer. I wonder why.
Alternatively: Americans are obsessed with building, so they whine about red tape; the Chinese are obsessed with grifting, so they pretend to build. But what has China built concretely? Rail for empty trains, and empty apartment blocks? Automated ports to ship Aliexpress gizmos to the Americans? This is all immaterial in the AI race. Where are their new supercomputers? Buying out consumer GPUs? Centralized collection of annotations to train foundation models, incentivized with Social Credit score (does it even work yet)? They have many levers to compensate for their hardware and expertise shortcomings. Which ones, exactly, have they pressed? They've only ever gone in the opposite direction – prohibiting tech giants from harvesting data, imposing regulations, forgoing opportunities.
They'd rather take an unsustainable loan, erect another concrete dildo, stuff it with pigs and Huawei snout recognition and pat themselves on the back for being innovative. All the while some pig-like official collects gold bricks in the basement of his overpriced siheyuan in the countryside. That's what Chinese building is like.
I may sound a little racist here. But the bigger issue is that Mainland China is so incredibly sheltered. They don't have the sense of what is possible, their culture is a tiny shallow hothouse for midwit takes. It's like Belarus or some other stale post-Soviet backwater; actually worse. This is true of their entertainment as well as of their tech and politics. I've tried to take them seriously for a while, and came to this conclusion. Ignoring China and assuming they won't do anything consequential nor retaliate in any meaningful way when Anglos are kicking them in the balls has consistently been the rational choice.
They've curtailed this strategy though, now it's about «the struggle for security» or something. Not like it'll work.
More options
Context Copy link