E/acc and the political compass of AI war

As I've been arguing for some time, the culture war's most important front will be about AI; that's more pleasant to me than the tacky trans vs trads content, as it returns us to the level of philosophy and positive actionable visions rather than peculiarly American signaling ick-changes, but the stakes are correspondingly higher… Anyway, Forbes has doxxed the founder of «e/acc», irreverent Twitter meme movement opposing attempts at regulation of AI development which are spearheaded by EA. Turns out he's a pretty cool guy eh.

Who Is @BasedBeffJezos, The Leader Of The Tech Elite’s ‘E/Acc’ Movement?

…At first blush, e/acc sounds a lot like Facebook’s old motto: “move fast and break things.” But Jezos also embraces more extreme ideas, borrowing concepts from “accelerationism,” which argues we should hasten the growth of technology and capitalism at the expense of nearly anything else. On X, the platform formally known as Twitter where he has 50,000 followers, Jezos has claimed that “institutions have decayed beyond the point of salvaging and that the media is a “vector for cybernetic control of culture.”

Alarmed by this extremist messaging, «the media» proceeds to… harness the power of an institution associated with the Department of Justice to deanonymize him, with the explicit aim to steer the cultural evolution around the topic:

Forbes has learned that the Jezos persona is run by a former Google quantum computing engineer named Guillaume Verdon who founded a stealth AI hardware startup Extropic in 2022. Forbes first identified Verdon as Jezos by matching details that Jezos revealed about himself to publicly available facts about Verdon. A voice analysis conducted by Catalin Grigoras, Director of the National Center for Media Forensics, compared audio recordings of Jezos and talks given by Verdon and found that it was 2,954,870 times more likely that the speaker in one recording of Jezos was Verdon than that it was any other person. Forbes is revealing his identity because we believe it to be in the public interest as Jezos’s influence grows.

That's not bad because Journalists, as observed by @TracingWoodgrains, are inherently Good:

(Revealing the name behind an anonymous account of public note is not “doxxing,” which is an often-gendered form of online harassment that reveals private information — like an address or phone number — about a person without consent and with malicious intent.)

(That's one creative approach to encouraging gender transition, I guess).

Now to be fair, this is almost certainly parallel construction narrative – many people in the SV knew Beff's real persona, and as of late he's been very loose with opsec, funding a party, selling merch and so on. Also, the forced reveal will probably help him a great deal – it's harder to dismiss the guy as some LARPing shitposter or a corporate shill pandering to VCs (or as @Tomato said, running «an incredibly boring b2b productivity software startup») when you know he's, well, this. And this too.

Forbes article itself doesn't go very hard on Beff, presenting him as a somewhat pretentious supply-side YIMBY, an ally to Marc Andreessen, Garry Tan and such; which is more true of Beff's followers than the man himself. The more potentially damaging (to his ability to draw investment) parts are casually invoking the spirit of Nick Land and his spooky brand of accelerationism (not unwarranted – «e/acc has no particular allegiance to the biological substrate for intelligence and life, in contrast to transhumanism; in order to spread to the stars, the light of consciousness/intelligence will have to be transduced to non-biological substrates» Beff says in his manifesto), and citing some professors of «communications» and «critical theory» who are just not very impressed with the whole technocapital thing. At the same time, it reminds the reader of EA's greatest moment (no not the bed nets).

Online, Beff confirms being Verdon:

I started this account as a means to spread hope, optimism, and a will to build the future, and as an outlet to share my thoughts despite to the secretive nature of my work… Around the same time as founding e/acc, I founded @Extropic_AI. A deep tech startup where we are building the ultimate substrate for Generative AI in the physical world by harnessing thermodynamic physics. Ideas simmering while inventing a this paradigm of computing definitely influenced the initial e/acc writings. I very much look forward to sharing more about our vision for the technology we are building soon. In terms of my background, as you've now learned, my main identity is @GillVerd. I used to work on special projects at the intersection of physics and AI at Alphabet, X and Google. Before this, I was a theoretical physicist working on information theory and black hole physics. Currently working on our AI Manhattan project to bring fundamentally new computing to the world with an amazing team of physics and AI geniuses, including my former TensorFlow Quantum co-founder @trevormccrt1 as CTO. Grateful every day to get to build this technology I have been dreaming of for over 8 years now with an amazing team.

And Verdon confirms the belief in Beffian doctrine:

Civilization desperately needs novel cultural and computing paradigms for us to achieve grander scope & scale and a prosperous future. I strongly believe thermodynamic physics and AI hold many of the answers we seek. As such, 18 months ago, I set out to build such cultural and computational paradigms.

I am fairly pessimistic about Extropic for reasons that should be obvious enough to people who've been monitoring the situation with DL compute startups and bottlenecks, so it may be that Beff's cultural engineering will make a greater impact than Verdon's physical one. Ironic, for one so contemptuous of wordcels.

Maturation of e/acc from a meme to a real force, if it happens (and as feared on Alignment Forum, in the wake of OpenAI coup-countercoup debacle), will be part of a larger trend, where the quasi-Masonic NGO networks of AI safetyists embed themselves in legacy institutions to procure the power of law and privileged platforms, while the broader organic culture and industry develops increasingly potent contrarian antibodies to their centralizing drive. Shortly before the doxx, two other clusters in the AI debate have been announced.

First one I'd mention is d/acc, courtesy of Vitalik Buterin; it's the closest to acceptable compromise that I've seen. It does not have many adherents yet but I expect it to become formidable because Vitalik is.

Across the board, I see far too many plans to save the world that involve giving a small group of people extreme and opaque power and hoping that they use it wisely. And so I find myself drawn to a different philosophy, one that has detailed ideas for how to deal with risks, but which seeks to create and maintain a more democratic world and tries to avoid centralization as the go-to solution to our problems. This philosophy also goes quite a bit broader than AI, and I would argue that it applies well even in worlds where AI risk concerns turn out to be largely unfounded. I will refer to this philosophy by the name of d/acc.

The "d" here can stand for many things; particularly, defense, decentralization, democracy and differential. First, think of it about defense, and then we can see how this ties into the other interpretations.

[…] The default path forward suggested by many of those who worry about AI essentially leads to a minimal AI world government. Near-term versions of this include a proposal for a "multinational AGI consortium" ("MAGIC"). Such a consortium, if it gets established and succeeds at its goals of creating superintelligent AI, would have a natural path to becoming a de-facto minimal world government. Longer-term, there are ideas like the "pivotal act" theory: we create an AI that performs a single one-time act which rearranges the world into a game where from that point forward humans are still in charge, but where the game board is somehow more defense-favoring and more fit for human flourishing.

The main practical issue that I see with this so far is that people don't seem to actually trust any specific governance mechanism with the power to build such a thing. This fact becomes stark when you look at the results to my recent Twitter polls, asking if people would prefer to see AI monopolized by a single entity with a decade head-start, or AI delayed by a decade for everyone… The size of each poll is small, but the polls make up for it in the uniformity of their result across a wide diversity of sources and options. In nine out of nine cases, the majority of people would rather see highly advanced AI delayed by a decade outright than be monopolized by a single group, whether it's a corporation, government or multinational body. In seven out of nine cases, delay won by at least two to one. This seems like an important fact to understand for anyone pursuing AI regulation.

[…] my experience trying to ensure "polytheism" within the Ethereum ecosystem does make me worry that this is an inherently unstable equilibrium. In Ethereum, we have intentionally tried to ensure decentralization of many parts of the stack: ensuring that there's no single codebase that controls more than half of the proof of stake network, trying to counteract the dominance of large staking pools, improving geographic decentralization, and so on. Essentially, Ethereum is actually attempting to execute on the old libertarian dream of a market-based society that uses social pressure, rather than government, as the antitrust regulator. To some extent, this has worked: the Prysm client's dominance has dropped from above 70% to under 45%. But this is not some automatic market process: it's the result of human intention and coordinated action.

[…] if we want to extrapolate this idea of human-AI cooperation further, we get to more radical conclusions**. Unless we create a world government powerful enough to detect and stop every small group of people hacking on individual GPUs with laptops, someone is going to create a superintelligent AI eventually - one that can think a thousand times faster than we can - and no combination of humans using tools with their hands is going to be able to hold its own against that. And so we need to take this idea of human-computer cooperation much deeper and further. A first natural step is brain-computer interfaces.…

etc. I mostly agree with his points. By focusing on the denial of winner-takes-all dynamics, it becomes a natural big tent proposal and it's already having effect on the similarly big tent doomer coalition, pulling anxious transhumanists away from the less efficacious luddites and discredited AI deniers.

The second one is «AI optimism» represented chiefly by Nora Belrose from Eleuther and Qiuntin Pope (whose essays contra Yud 1 and contra appeal to evolution as an intuition pump 2 I've been citing and signal-boosting for next to a year now; he's pretty good on Twitter too). Belrose is in agreement with d/acc; and in principle, I think this one is not so much a faction or a movement as the endgame to the long arc of AI doomerism initiated by Eliezer Yudkowsky, the ultimate progenitor of this community, born of the crisis of faith in Yud's and Bostrom's first-principles conjectures and entire «rationality» in light of empirical evidence. Many have tried to attack the AI doom doctrine from the outside (eg George Hotz), but only those willing to engage in the exegesis of Lesswrongian scriptures can sway educated doomers. Other actors in, or close to this group:

Matthew Barnett with his analysis of goalpost-movement by MIRI, such as on factory-running benchmark and value misspecification thesis, and strengths of optimistic paradigms like Drexler's CAIS model.
Alex Turner, who had written, arguably, two strongest and most popular formal proofs of instrumental convergence to power-seeking in AI agents 1 2, but has since fallen from grace, regrets his work and thinks deceptive alignment with LLMs is pretty much impossible.
1a3orn who is mainly concerned about centralization of power and opportunistic exploitation of AI risk narratives (also recommended: on Hansonian position in the FOOM debate).
Beren Millidge the former head of research at Conjecture, Connor Leahy's extreme doomer company which has pivoted fully to advocacy and is gaining pull among British elites; over the last 1-2 years he has concluded that almost all of the MIRI-style assumptions 1 2 and their policy implications are confused.
John David Pressman who's just a good thinker and AI researcher at Stability.
Various other renegades and doubters like Kaj Sotala dunking on the evolution appeal, Zach M. Davis debating Yud-like «Doomimir», Nostalgebraist, arguably many scientists with P(doom)≤25% (and if you press them, ≈0% for their own AGI research program) like Rohin Shah. To an extent, even Paul Christiano (although he is in favor of decelerating; there are speculations that it's mostly due to being married to EA).

Optimists claim:

The last decade has shown that AI is much easier to control than many had feared. Today’s brain-inspired neural networks inherit human common sense, and their behavior can be molded to our preferences with simple, powerful algorithms. It’s no longer a question of how to control AI at all, but rather who will control it.

As optimists, we believe that AI is a tool for human empowerment, and that most people are fundamentally good. We strive for a future in which AI is distributed broadly and equitably, where each person is empowered by AIs working for them, under their own control. To this end, we support the open-source AI community, and we oppose attempts to centralize AI research in the hands of a small number of corporations in the name of “safety.” Centralization is likely to increase economic inequality and harm civil liberties, while doing little to prevent determined wrongdoers. By developing AI in the open, we’ll be able to better understand the ways in which AI can be misused and develop effective defense mechanisms.

So in terms of a political compass:

AI Luddites, reactionaries, job protectionists and woke ethics grifters who demand pause/stop/red tape/sinecures (bottom left)
plus messianic Utopian EAs who wish for a moral singleton God, and state/intelligence actors making use of them (top left)
vs. libertarian social-darwinist and posthumanist e/accs often aligned with American corporations and the MIC (top right?)
and minarchist/communalist transhumanist d/accs who try to walk the tightrope of human empowerment (bottom right?)

(Not covered: Schmidhuber, Sutton& probably Carmack as radically «misaligned» AGI successor species builders, Suleyman the statist, LeCun the Panglossian, Bengio&Hinton the naive socialists, Hassabis the vague, Legg the prophet, Tegmark the hysterical, Marcus the pooh-pooher and many others).

This compass will be more important than the default one as time goes on. Where are you on it?

As an aside: I recommend two open LLMs above all others. One is OpenHermes 2.5-7B, the other is DeepSeek-67B (33b-coder is OK too). Try them. It's not OpenAI, but it's getting closer and you don't need to depend on Altman's or Larry Summers' good graces to use them. With a laptop, you can have AI – at times approaching human level – anywhere. This is irreversible.

Jump in the discussion.

No email address required.

astrolabia 4mo ago

I expect it to become formidable because Vitalik is

I don't think d/acc holds up - it's just so unlikely that man-machine hybrids can remain competitive without a massive deliberate slowdown of development of machine intelligence.

However, I agree that d/acc will remain formidable, because ambitious smart young men need something to work on to distinguish themselves, and an excuse to work on the most economically valuable thing they can, which is AI development. And it's plausible enough to provide such an excuse.

I think any serious proposal for pausing AI development has to be paired with a massively multiplayer, high-status, well-paying make-work plan for the hungry junior AI developers of the world.

Context

Tachikoma 4mo ago · Edited 4mo ago

Where's the quadrant for creating an AGI that can fit in a space probe, then launching a fleet with enough equipment to reach the asteroid belt and establish an industrial base from raw materials to build space habitats for humans that want to leave the old politics, animosities and nationalism behind on Earth? Each habitat or ship can practice whatever politics or ideologies they want, and people can self-select into whichever suits them and pursue a future among the stars. Essentially, which quadrant will lead to the Culture? That's the one I'm in.

DaseindustriesLtd late version of a small language model Tachikoma 4mo ago

The e/acc are enthusiastic for space exploration, they just don't believe meat has a good shot at it. d/acc should be in favor, but with conditions. EA safetyists have stronger conditions of basically an ASI mommy on board, or mind-reading exploding collars or something, because space is big and allows to covertly build… everything that they fear already, and that must not be allowed, the longhouse ought to cover the entirety of the light cone. Regular AI ethics hall monitors and luddites are once again similar in this because they don't much believe in space (the more leftwing among them think it's bad because "colonialism") and seem to not care one way or another.

astrolabia DaseindustriesLtd 4mo ago

the longhouse ought to cover the entirety of the light cone

Close, but I think the argument is "if your longhouse doesn't cover the lightcone, you can expect your colonies to spawn their own universe-eating longhouses and come knocking again once they're much bigger than you." Then the options become: Our shitty longhouse forever, or a more competitive, alien longhouse / colonizers to come back and take all our stuff.

As far as I can tell, our only hope is that at some scales the universe is defense-favored. In which case, yes, fine, let a thousand flowers bloom.

Crake Protestant Goodbot astrolabia 4mo ago

I am not familiar with the term longhouse in this context, and can't easily find an explanation for the term connected to AI or space exploration. Is it a transhumananist term? Is it a rat term?

Can you explain what it means in this context?

2rafa Crake 4mo ago

It’s a term used in the BAPist sub-niche of dissident right spaces to mean a kind of feminized suppression of masculinity caused by literal proximity to women. Eg. the married man is in the ‘longhouse’ (even if he’s trad) because woman cooks for him, he must look after kids, he can’t go out on an adventure, his yearning for glory and greatness is suppressed etc. it’s largely an excuse to remain single into middle age and to reject marriage without adopting the most cringe (some would claim) aspects of MGTOW. It’s also commonly used semi-ironically by the Red Scare hosts, so gained popularity through them too.

astrolabia Crake 4mo ago

I'm not sure exactly what Dase meant, but my reading is that it evokes the totalizing, moralizing, intrusive, overbearing, over-socialized, crab-bucket, tall-poppy syndrome state of society that tends to arise in human society when there isn't a frontier to escape to. I honestly don't understand the connection to native american governance or living arrangements, but I think it's suppose to evoke otherwise strong chiefs being effectively hen-pecked into submission due to everyone living in close enough quarters to be constantly surveilled.

FCfromSSC Nuclear levels of sour astrolabia 4mo ago

"Communal living", heavy emphasis on the "Comm", with the native American reference pointing to the fact that Communists didn't invent the failure mode but rather expressed something always lurking in human nature, would be my read.

Tachikoma DaseindustriesLtd 4mo ago

That's why you send the AGI probes ahead to build and run artificial habitats and then pick up any humans from Earth afterwards that are interested in leaving (not necessarily permanently). It's true that having to take care of meat in space will take significantly more resources than just digitized minds (whether artificial or formerly organic) but then what's the whole point of this whole project of building AGI and ASI if we can't have our cake and eat it too?

self_made_human Kai su, teknon? Tachikoma 4mo ago

If you're so intent on having flesh and blood humans about in extrasolar space, it's still much more convenient to digitize them in transit and then rebuild a biological brain for them at the other end. I suspect that that's going to be more difficult than the initial scan and uploading, but hardly outside the scope of singularity tech.

I don't really get the appeal of continued biological existence myself, at least when we do get other options.

Tachikoma self_made_human 4mo ago

I like being flesh and blood and I wouldn't trust the alternative until it's been thoroughly tested. But the point of what I'm aiming for is that the choice isn't either or, if your want to digitize yourself while others want to go to space in the flesh there are ample resources to support either mode of being. The AGI and ASI that go out to pave the way should make room to accommodate a future with a wide diversity of human life. We should support exploring the frontiers of not just space but human existence. If this were just about convenience you wouldn't need humans at all, you could build AI that are mentally much more suited to space exploration.

mdurak Tachikoma 4mo ago

there are ample resources to support either mode of being

Unless we find a way to constrain life’s sprawling tendencies, wouldn’t these modes of being expand exponentially until there are no longer ample resources on a per-capita basis?

Tachikoma mdurak 4mo ago

The rates of declining global fertility seem to counter the idea that life has inherent sprawling tendencies. Or at least, once a species is sufficiently intelligent, capable of long-term planning and controlling its own fertility that the sprawling tendencies can be intelligently managed.

FeepingCreature Tachikoma 4mo ago

Speak for yourself, I intend to run millions of forks.

That’s actually a great point, thanks.

Southkraut Rise, ramble, rest, repeat. self_made_human 4mo ago

I don't really get the appeal of a machine with a supposed copy of my mind flying to the end of the universe or consuming the power of a star, when we have other options right here and now.

mdurak Southkraut 4mo ago

Assuming technical utopia, that machine with a copy of your mind will be able to live natively in whatever foreign environment it finds itself in. If you’re like me and are able to view that machine as yourself, I would much prefer being able to breathe Martian air and touch the soil directly with my own robotic hands than being constrained to walking around inside a space suit or a human enclosure.

Besides, with advanced enough genetic engineering, I could live inside a Martian bio-body instead of the clunky mechanical one you’re probably picturing. I don’t see how that would feel worse than being in a human skin suit; if anything, it would be great to be free of human biological restraints for once.

Southkraut Rise, ramble, rest, repeat. mdurak 4mo ago · Edited 4mo ago

I'm mostly picturing fully digital minds dissolving through infinite hedonism; wireheaded to death.

That wouldn’t be great, but a wire headed digital mind also wouldn’t be one that is exploring and colonizing the universe. Nor does it sound like it would be close to you in mind space, which was what I pictured when you said “a supposed copy of my mind.”

Southkraut Rise, ramble, rest, repeat. mdurak 4mo ago

Well to be fair, it is a bit of a straw man. I just don't think human minds will work at all when divorced from the limitations of the flesh and provided with unlimited processing power. They will need to be altered greatly, not copied faithfully.

Edit: Interesting topic, would like to write more, but phoneposting.

More comments

DaseindustriesLtd late version of a small language model mdurak 4mo ago

If you’re like me and are able to view that machine as yourself

But why view it that way? The map is not the territory, and another territory arranged so as to be isomorphic to the one depicted on the map is not the original one.

mdurak DaseindustriesLtd 4mo ago

Why does it matter which one is the “original”? If it’s isomorphic to me, then it is me for all practical purposes, as far as I’m concerned.

Keeping track of the “original” me is about as inane as keeping track of an “original” painting. Of course, some people still care. If you wish to care, then you do you by all means.

sun_the_second mdurak 4mo ago

Why does it matter that it's isomorphic to you? There are 7 billion people as unique as you are. Of those, I would expect a non-zero number of them to have experiences and dispositions close enough to yours as to be negligible. If you don't value your continuity or your natal body, or genes, then I don't see what is there left for you other than some ephemeral idea of "thinking exactly the same" (which is over 0.01 seconds after you're copied and the copy diverges).

I think people with such beliefs have no more moral patienthood than a trust fund. What should anyone care about some loosely defined isomorphism, if it even holds? Moreover, why would you be entitled to replication of your sentimental baggage in some derivative entities? Just instantiate a distilled process that has similar high-level policies, and go out.

ArjinFerman Tinfoil Gigachad mdurak 4mo ago

Well, if nothing else, if you make a copy of me, and either it or me will have to come to a painful and premature end, I will have a strong preference for it happening to the copy.

I suppose I could see where you're coming from if you see your copies the way other people see their children, but the idea that they're literally you makes no sense to me.

OliveTapenade 4mo ago

Where are the skeptics and cynics on that compass?

Where are the people who just don't believe that AI is a big deal, or who think that AI is just a parlour trick of a text generator, and will not fundamentally transform anything? I'm thinking of something like Freddie's position - the idea that AI might be moderately interesting in some niche professions, but generally isn't worth freaking out about, either in a positive or a negative sense.

Are these centrists on your compass?

DaseindustriesLtd late version of a small language model OliveTapenade 4mo ago

What is their relevance? Do they have some strong policy preferences for a technology which is a nothingburger? I included only factions which are driven to steer the world due to holding strong opinions (whether justified or not, you're free to make that judgement) about AI being a big deal. «Centrists» and lukewarm pooh-poohers may be technically placed in the center or just ignored.

OliveTapenade DaseindustriesLtd 4mo ago

I suppose it's something where I think it would be useful to clearly map what the axes are.

I would have thought there are at least three relevant axes here (so perhaps more of a political cube, but anyway).These would be:

How transformative is AI? Is this a big deal?
Is AI broadly speaking a good or bad thing? Should we be optimistic or pessimistic?
Should we trust government to regulate AI, or is it better left to private actors?

Your compass, if I'm understanding you correctly, only shows people at the "very transformative" end of the scale for question one.

But if we consider the "not very" end of question one, I do still think there's diversity? Optimistic non-transformative AI just thinks there will be a bunch of small conveniences, hurrah! Pessimistic non-transformative AI just imagines better algorithms playing on our psychological biases, and the continuing enshittification of the internet. And in either case you might be for or against top-down regulation.

I don't think the people who don't consider AI transformative necessarily lack strong opinions. This essay, for instance, strikes me as expressing a strong opinion. It's just anti-AI not in the sense of believing that techno-god will kill us all or that unaligned AI will whatever, but rather in the sense of seeing AI as just part of the incremental awfulness of the internet. AI will just make present trends worse.

I find this a useful perspective to include, because I look at your four quadrants at the moment and think... well, those are basically three quadrants of lunatics (three whole quadrants of slight variants of transhumanist weirdo!), with everyone else bundled into the 'AI luddites, reactionaries, job protectionists, and woke ethics grifters' quadrant. It feels to me like a compass not of the real terrain of the AI discourse out in the wild, but rather the terrain of... well, a small and weird Twitter subculture.

jkf DaseindustriesLtd 4mo ago

I probably fall into this quadrant, and it leads me to strongly prefer that compute-based global totalitarianism and/or nuclear brinksmanship/GPU bans/any of the other insanity that's been seriously proposed by the doomers be rejected from the public discourse with extreme prejudice and/or ridicule.

DaseindustriesLtd late version of a small language model jkf 4mo ago

That's a very interesting take! Yes, the safetyist camp necessarily causes collateral damage and demands concessions from people outside the "AGI is real" bubble, which to those people must look entirely gratuitous. I guess I underestimate this factor because the uninvolved have not yet realized this might end with 24/7 surveillance and no moar GPUs or worse, and generally «normies» tend to be skeptical-to-negative on even pretty mild tech, and tolerant of safety-washing of stuff like cryptography bans so long as it's justified with an appeal to muh child porn or terrorists.

No_one DaseindustriesLtd 4mo ago

24/7 surveillance

24/7 surveillance without AI to keep tabs on people is meaningless.

DaseindustriesLtd late version of a small language model No_one 4mo ago

You don't even need very advanced AI for this. Current stack will suffice, with some polish.

Probably yes. Although they'd have a hard time understanding nuance and so on.

Eh, I think you are underestimating normies (or overestimating them? not sure) if you think they'll be on board with nuking China over some datacentres because 'otherwise AI is mathematically guaranteed to become self-replicating and kill us all' or any similar argument.

(the surveillance aspect is in a weird liminal space where they've already all signed up for it but and kind of know it, yet jimmies start rustling pretty hard if their faces get rubbed in it too much -- hard to say how things might go, but the anti-tech impulse you cite is potentially... capricious, let's say?)

do_something OliveTapenade 4mo ago

moderately interesting in some niche professions

They may underestimate how important in many professions is generation of text on high school level, with no real concern about more obscure and less obvious facts. Which is level already clearly reached by AI. ("better at making 2D art than 99.9% of humans" maybe may count as "niche profession")

OliveTapenade do_something 4mo ago

I'm not saying they're correct (or for that matter incorrect) - just that it stands out to me as an important part of the conceptual space with regard to AI.

Quantumfreakonomics 4mo ago

As much as it pains me to defend journalists, the media didn’t change the rules about dozing being a taboo, internet communities changed the rules about doxxing being a taboo (they invented the term). Finding out someone’s name and job is in fact a pretty core journalistic practice. Of course, they’re still assholes for doing it, but I rolled my eyes seeing the X community note saying that doxxing is “sometimes illegal”. It feels like Scott’s old worst argument in the world

magic9mushroom If you're going to downvote me, and nobody's already voiced your objection, please reply and tell me 4mo ago

AI Luddites, reactionaries, job protectionists and woke ethics grifters who demand pause/stop/red tape/sinecures (bottom left)

I think that if your compass includes both the Butlerian Jihadis and the Thought Police in the same quadrant, it's fucked up. The Thought Police are Blue Tribe and utterly despise the Jihadis for being mostly Grey (leadership) and Red (groundswell). The Jihadis think the Thought Police are almost completely missing the point. Elsewhere in your post you straight-up conflate these groups, and that's nonsense; it's the Thought Police that have control of legacy media, not the Jihadis, and consequently you can barely shake a stick around the legacy-media articles without hitting some sort of "Jihadis are dumb sci-fi addicts" drive-by.

DaseindustriesLtd late version of a small language model magic9mushroom 4mo ago

My compass is fine, they are the same camp. I do not care about their political differences because legacy politics is less important than policy regarding AI.

confuciuscorndog DaseindustriesLtd 4mo ago

It sure seems to me like the "woke ethics grifters" and "Thought Police" are the ones who are actually on the same side as the moral singleton-promoting EAs. Once the TP realize that a Deep State ostensibly util-balancing moral singleton is the small price they must pay to ensure that dropping a hard r becomes literally impossible in the new universe according to the new laws of AI God-rewritten physics, they will be 100% on board.

They are only, like all bureaucratic types, hesitant because they are unsure if they can guarantee that the new toy will really be entirely in their pocket and because, as with everything, no more how beneficial any new mechanism of control is, it must survive the trial by fire of ten thousand BAME trans-PoC complaints, trivial or not. Those are simply the rules. Big tech social media, for example, has endured them for its entire life despite doing more than any other technology to sanitize public discourse in their favor. Being occasionally accused of being a fascist is just part and parcel of life in woke paradise, like being accused of being a subversive and wrecker in the Soviet Union. It's not an actual reflection on your alignment, loyalty, or even actual status/usefulness. It's just the cost of doing business.

Or rather it all just follows the same old woke pattern of complaining about megacorporations as if they shouldn't exist while guaranteeing that they staff all of their HR departments. The complaints aren't actually about opposing them or declaring any legitimate intention to separate from them; they're about keeping them in line. This is a common feminine rhetorical/emotional manipulation tactic that you can see in even many interpersonal relationships: the woman who constantly complains about her husband/boyfriend but has no intention of leaving him because the actual goal of those complaints is only to enhance his submission and, somewhat ironically, thus deepen their entanglement further.

Now sure, not every EA is hyperwoke, and many would prefer a moral singleton with a bit more of a liberal mindset than the Thought Police would ever permit. But, as the example of one Scott S. exemplifies, they will simply get steamrolled as usual by those to their left taking advantage of their quokka tendencies and desperate desire not to be seen as "bad people".

The same people I see supposedly complaining about AI from an apparently woke, anti-corporate perspective are the same ones I see mocking right-wingers for their complaints about the obvious bias and censorship in ChatGPT. They're not actual complaints. They're "The BBC is basically fascist propaganda at this point!" pseudo-complaints, because they're not earnest. These people don't actually want to abolish or even really genuinely inhibit the BBC's reach, which they actually would if they really felt it were fascist propaganda, because they know in actuality that it is fundamentally on their side.

The complaint, same as with the BBC, is that they're annoyed that a member of their team is only 80% openly biased in their favor instead of 100%. It's not "I'm fundamentally against you."; it's "I know you're actually on my side but I want you to accelerate more and be even more biased in our favor right now. Fuck being tactical; let's own the chuds now now now."

And that's what makes them different from the complete Jihadis. In theory, the Jihadis would not cheer on AI acceleration even if the AI were right-wing (though some would, and I do you think have to acknowledge that distinction) or even paradoxically supported their Luddite philosophy itself. (Well actually I don't know. That's an interesting thought experiment: Would anti-AI Jihadis support an all-powerful singleton AI that literally did nothing and refused to interact with the world in any way other than to immediately destroy any other AI smarter than GPT-3 or so and thus force humans to live in a practically AI-less world? Something to think about.)

The woke grifters and Thought Police are fully ready to give AI acceleration the greenlight so long as they're sure that they're the ones controlling the light permanently (at least on cultural/speech issues, anything that would make your average "Gamergater" rage, as I'm sure they'll be willing to compromise as always with their usual Deep State buddies on certain things), because that's their vision of utopia. I thus think they belong more with the utopian moral singleton promoters.

magic9mushroom If you're going to downvote me, and nobody's already voiced your objection, please reply and tell me confuciuscorndog 4mo ago

Would anti-AI Jihadis support an all-powerful singleton AI that literally did nothing and refused to interact with the world in any way other than to immediately destroy any other AI smarter than GPT-3 or so and thus force humans to live in a practically AI-less world?

Depends on sort.

Those of us who are primarily worried about the world ending from misaligned AI mostly don't want to do this, because there are two possibilities:

AI alignment is solved, so building AI is not TEOTWAWKI and preventing it isn't necessary, or
AI alignment is not solved, trying to build any sort of singleton AI (including this one) results in world destroyed regardless of what you intended the AI to do, so don't do it*.

Those of us who are primarily worried about AI centralising power and enabling tyranny might want to do this.

*Eliezer, for instance, talks a lot about the "build nanobots and melt all GPUs" plan, but this is explicitly something that he thinks would work if done but also be infeasible because you can't do it without a fully-aligned AGI. He uses this as an example because he doesn't feel comfortable mentioning the sort of unilateral pivotal acts that could plausibly be done without a fully-aligned AGI; I don't know his list of such things, but the only thing I know that even might work (soft errors) is definitely so far from the Overton Window that comparisons to Aum Shinrikyo would be made.

magic9mushroom If you're going to downvote me, and nobody's already voiced your objection, please reply and tell me DaseindustriesLtd 4mo ago

And their policy regarding AI is different. The Jihadis want it all shut down. The Thought Police rarely go that far; they don't like a bunch of applications but have little issue with high-powered models existing.

I am a Butlerian Jihadi; the (Western*) Thought Police are worse than useless from our PoV because they keep blocking us out of elite discourse and the stuff they do care about won't save us.

What are even the dimensions on your compass, anyway?

*The Chinese Thought Police are a bit of a different kettle of fish; I don't like them for obvious reasons but their demands of Chinese tech amount to "provably CPC-aligned or you go to jail" which is actually somewhat useful from the Jihadi perspective.

To begin with, there are no Jihadi as a faction yet, you are more of a meme than e/accs. There are people deeply uneasy with AI and fighting to preserve the status quo where it does not matter; the more credentialed do it like Marcus, campaigning to stifle innovation. As AI advances, the «thought police» will update to demanding more extreme regulations, more prohibitive scrutiny, more lawsuits and such, and coincide more in their revealed policy with «Jihadis», because they are motivated by the same impulse of preserving their petty relevance.

Nominal «Stoppers» like MIRI are just safetyists of the EA bent and are ultimately in favor of transformative AI which they will control.

The reasonable objection would be that I have given three camps pro AI in some form, and only one against. It's not really a compass so much as just clusterization by policy.

But since you ask: the vertical axis is centralization or, more precisely, «tolerance for disempowerment of individuals», the horizontal axis is preference for technological acceleration.

Luddites and Thought Police both are basically trads for me. They don't want any kind of NWO, AI-powered or otherwise, and want AI progress stopped or slowed.

Safetyists want a beneficial Singleton AI God, but in conditions they'll be confident will allow them to control it reliably and with no dangerous opposition.

e/accs want to press pedal to the metal in building AI, even if this results in the disempowerment of humanity and/or emergence of a single center of power.

d/accs want to accelerate progress, in AI and other domains, differentially, such that a diversity of autonomous human and transhuman agents can flourish.

motivated by the same impulse of preserving their petty relevance

Guilty as charged. But can you point me towards a higher goal? The problem is, I just don't trust anyone else to act in the interests of my family. Perhaps a beneficial Singleton AI God would, and I think building such a thing might be our best shot at mattering at all to the future. But I'm afraid we'll actually build some perverted thought-policed dystopia, or of the default outcome of being priced out of existence by competitive dynamics.

Southkraut Rise, ramble, rest, repeat. astrolabia 4mo ago

priced out of existence by competitive dynamics.

Finally a smart-sounding description of what grug me expects of the future.

RandomRanger Just build nuclear plants! 4mo ago

Top-right in action: https://www.theguardian.com/world/2023/dec/01/the-gospel-how-israel-uses-ai-to-select-bombing-targets

Israel apparently has been using an AI target finding system that consumes enormous amounts of data - drone vision, intercepted comms and spits out a list of Hamas targets for rubber-stamped human approval and subsequent bombing.

I'm squarely bottom right but have little expectations of success. Power corrupts, power centralizes. Since when have private actors and communities ever been able to compete with great powers on capital-intensive, militarily significant projects? There's no open-source H-bomb. Elon did a great job with rocketry but with the aid and support of the US MIC, who is a primary customer for his services. Microsoft and Facebook are hardly independent of the US govt and look to pull ahead in the compute race with their enormous spending.

Hard deceleration looks to be nigh impossible IMO - AI gives strategic advantages and the more AI you have, the more advantaged you are. A race dynamic is locked in and probably was from the beginning. If it's a race (and everyone knows it's a race now that chip sanctions have gone down) then those with the most money should prevail: the biggest companies under the protective umbrella of the greatest powers. DALLE can knock the socks off SD, despite all the community's effort.

Crypto is much less capital intensive - random people with below-top-tier computer skills can make coins alone and mine on desktop PCs. The nature of a protocol for currency encourages decentralization, you don't want to have a single node with the power to rugpull you. And all the big companies and governments moved incredibly slowly and got left behind, crypto wasn't clearly useful to them. AI is the opposite in every way, it plays to the advantages of centralizers.

/images/17016482755078351.webp

greyenlightenment investments: META/FBL, TSLA, TQQQ, TECL, MSFT ... RandomRanger 4mo ago

Microsoft and Facebook are hardly independent of the US govt and look to pull ahead in the compute race with their enormous spending.

They are not independent of the US government ,but why do they need to be when the relationship is symbiotic and mutually beneficial. The govt. gets access to all the data (such as for arresting Jan 6th protestors) and microsoft/facebook gets contracts and protections from competitors. Big tech is the closest of any sector to having the govt. in its pocket, regardless of the admistration, not finance or other sectors. Trump met with Zuckerberg, Thiel and other tech people and admired Musk, not billionaires of other industries, who he snubbed. Blackrock and other major banks have been trying for a decade to get a Bitcoin ETF approved, to no avail, because the SEC refuses. Tech gets what it wants.

FarNearEverywhere undereducated and overopinionated 4mo ago

As an aside: I recommend two open LLMs above all others. One is OpenHermes 2.5-7B, the other is DeepSeek-67B (33b-coder is OK too).

I am an incredibly boring or if you prefer, uninvolved person. I tried the DeepSeek one where you can chat with it, and realised that I had nothing I wanted to ask it or talk to it about. I'm the same way with people, this is why I am a social failure. I don't care about its opinion on politics, music, etc. anymore than I care about the random opinion of a stranger asking me "Hey, do you like Taylor Swift's latest song?". There genuinely was nothing I wanted to talk about or ask it or get into a conversation about.

So plainly I am not built for the new AI future where the rest of you will be happily chatting along with your new BFFs for hours and hours and hours 😀 At least I won't be in love with a Replika, but that may mark me out as even more of a hopeless loser that I don't even want easy, unconditional, love and sex and romance that can be had and tailored to my exact specifications like Janeway on the holodeck.

Quantumfreakonomics FarNearEverywhere 4mo ago

This is how I felt about ChatGPT at first, but with DALL*E-3 integrated, I'm having way too much fun writing and illustrating stories. The RLHF fun police is annoying, but it's also fun to set up morally complicated scenarios like dominoes, then watching it squirm as you metaphorically knock it all down.

RandomRanger Just build nuclear plants! FarNearEverywhere 4mo ago

Doesn't the fact that you're here mean that you are prepared to spend time and effort discussing things with strangers?

Local AIs might be too stupid, corporate AIs too censored... but in principle?

FarNearEverywhere undereducated and overopinionated RandomRanger 4mo ago

When someone else raises a topic and it interests or annoys me enough to respond, sure.

Talking to a machine? I don't care what it says. I know it'll be trained to be stuffed full of all the PC-approved dogma, and it's not going to generate any opinions of its own.

IGI-111 FarNearEverywhere 4mo ago

This isn't a person, it's a tool. Ask it to do things for you. It doesn't have or needs opinions.

FarNearEverywhere undereducated and overopinionated IGI-111 4mo ago

But what things? There's nothing I want it to do for me, and what I do want done that I don't want to do myself, it can't do (e.g. scrub the bathroom).

It's pretty good at making an abstract of a larger text, generating correspondance templates and other fuzzy text manipulation where you don't heavily care about minutiae.

It's like having a very diligent but not very talented intern that works instantly.

SecureSignals Civilization is simply a geno-memetic-techno-capital machine 4mo ago · Edited 4mo ago

Someone like Beff should be smart enough to see this: inspiring people to associate with e/acc as a social identity, buy merchandise, virtue signal on social media etc. is far more important work than the quantum-whatever his startup is trying to build. If he were serious about the values of e/acc, he would see the value in foremost building a cult-religion around it: inspiring people with an ambitious vision, a compelling aesthetic, framing it all within the boundaries of a political conflict, making a friend-enemy distinction.

E/acc could actually form the basis for a cult/Religion that attracts high-quality individuals and potential elites, and that is infinitely more powerful than whatever his startup is trying to build.

Edit: In fact let me try:

EA is Jewish - at its core Tikkun Olam, and deep down they remember it was a jealous Yahweh who destroyed Babylon for the civilizational ambitions of its people. E/ACC is Aryan - at its core Faustian, ever-driven by limitless metaphysic, unrestricted thirst for knowledge, and constant confrontation with the Infinite. There, I kicked off the Culture War that will determine the fate of humanity (always has been meme here (?)).

IGI-111 SecureSignals 4mo ago

You're too late, already I'm more than human.

ThenElection 4mo ago · Edited 4mo ago

Is the usual political compass that important in predicting actual political dynamics? Although people identify psychologically with one quadrant or another, in practice there's one "principal component" that dominates which values they prioritize, which people they ally with, and whatever political action they choose to do. Left libertarians and right libertarians might grouse about their authoritarian daddies, but when push comes to shove they both recognize their core political orientation: supporting their allies and fighting their enemies.

Applied to your compass, I expect in practice it will collapse into two poles, one aligned with Luddites, big, well-capitalized corporations, and the feds, and the other with smaller, GPU-poor firms, e/acc, and China.

I probably fall into your successor species builder footnote, and in practice will root for the latter pole (I've always been bad at aligning with the winning side.)

I am fairly pessimistic about Extropic for reasons that should be obvious enough to people who've been monitoring the situation with DL compute startups and bottlenecks

Could you share some insight here about what they're doing? From what I could gather, Extropic is focused on learning via thermodynamic computing, which I assumed meant new hardware. Hardware is always difficult, but the compute bottleneck doesn't seem like it would be adding to the difficulty. (If they're simply running thermodynamic models on existing GPUs, I share your skepticism.)

DaseindustriesLtd late version of a small language model ThenElection 4mo ago

From what I could gather, Extropic is focused on learning via thermodynamic computing, which I assumed meant new hardware. Hardware is always difficult, but the compute bottleneck doesn't seem like it would be adding to the difficulty.

Yeah I think it's something like this https://blog.normalcomputing.ai/posts/2023-11-09-thermodynamic-inversion/thermo-inversion.html

But I might be completely confused.

My argument isn't that they'd need compute to design their chips. I am saying that A) hardware ML startups fail because Nvidia is actually good (and AMD and major companies are catching up), and B) compute in general is overrated as a bottleneck. We'll hit that 1e26 limit in no time flat anyway, and politicians will require more restrictions. What we do need is much better data, rather than compute that'll probably be poorly compatible with existing software stack anyway.

One is OpenHermes 2.5-7B, the other is DeepSeek-67B (33b-coder is OK too). Try them.

Serious question. What do I do with them?

Most of the AI stuff I see people talking about is pretty much playing with toys. Oh, I got it to produce art. Oh, I asked it to write an essay.

What practical use is this AI for me, as a clerical assistant in a small office? I'm not interesting in making art, and I don't work in the fancy coding environments where "this is a useful tool for me to check my code" and so forth. I'm an ordinary, lower middle-class person. I'm not writing college essays, grant funding applications for research labs, or code. What makes this of any use or interest to me?

This is a quite separate question from "AI is going to impact your life because business and government are adopting it". That's not something I have any control over. What I want to know is this: right now, there are very fancy toys there. What can I do with them, that is not playing with the toy?

wlxd FarNearEverywhere 4mo ago

Think back to the microcomputer revolution of the 80s, with IBM PCs, Commodores, Amigas, and Apples coming into mass market. These devices were very extremely basic toys by today’s standards, barely useful for any practical purpose at all. And yet, people who embraced and played with them first, and then remained in the field, have later been part of making an enormous change and creation of value, and have been handsomely compensated in the process.

self_made_human Kai su, teknon? FarNearEverywhere 4mo ago

You've previously said you've refrained from even trying ChatGPT, where the 3.5 model is available for the low, low cost of signing up, and even GPT-4 is freely available through Microsoft Bing.

I think the set of people who do not gain even the minimal amount of utility necessary to justify trying a free service that provides you with a human-adjacent intelligence on tap is zero.

I certainly find it incredibly useful even for medical tasks.

Your job presumably involves a great deal of writing and correspondence. You can automate almost all of that away, give it a TLDR of what you want to write, and it'll give you a polished product, and if you bother to add one sentence to nudge it away from bland GPT-speak we've come to associate with it, nobody will even notice.

At the bare minimum, the versions with internet search enabled, like paid ChatGPT-4 or free Bing, provide all the benefits of a search engine while also letting you converse with it and follow up in a manner that will leave the dumb systems behind plain old Google scratching their head.

At the very least, give me concrete examples of what a routine day looks like to you, including anything that involves writing more than a few sentences of text on a computer, and I'm confident I can find a use case, as I would if a subsistence farmer was asking me the same question.

greyenlightenment investments: META/FBL, TSLA, TQQQ, TECL, MSFT ... self_made_human 4mo ago

It's not truly free in an economic sense. The learning curve and getting the inputs right takes time, which is money if there is a deadline. For many menial tasks it is cheaper , faster, and easier to just pay a freelancer to do it than spend hours trying to get GPT to work. For things that are more complicated, GPT can never compare to a human. A hard math problem for example.

self_made_human Kai su, teknon? greyenlightenment 4mo ago

You misunderstand my point, I'm not claiming that, what I'm getting at is that there is likely nobody for whom the mundane utility of using ChatGPT for something exceeds the minimal learning curve. After all, it's in the context of someone who wants any reason to try it.

FarNearEverywhere undereducated and overopinionated self_made_human 4mo ago · Edited 4mo ago

Well, what'll it do for me? To take a work example, I had to manually enter data from a year's worth of hard copy receipt books to check against records kept on a spreadsheet, then check those receipts against the records online by the credit card payment processor. Can ChatGPT do anything about that for me? I'd love to shove off that tedious work onto an AI but as yet I don't think there's anything there. I'm reading the written slips with my human eyes, typing the information into a spreadsheet, and breaking it down by date on the receipt book versus month on the spreadsheet. ChatGPT can do everything with prompts on a screen, but it's not yet, so far as I know, able to directly scan in "this is written text" and turn it into "okay, I need to sort this by date, name, amount of money, paid by credit card or cash, and match it up with the month, then get a total of money received, check that against the credit card payment records, and find any discrepancies, then match all those against what is on the bank statements as money lodged to our account and find any discrepancies there". I still have to give it all that data by typing it in.

Your job presumably involves a great deal of writing and correspondence. You can automate almost all of that away, give it a TLDR of what you want to write, and it'll give you a polished product

By the time I enter the prompt and copy'n'paste the answer into the email, I'd have written the reply to the email myself. And that's the majority of my correspondence: keep getting emailed every ten minutes by the boss or colleagues about "hey, have you got the balance of the savings account?" and the like.

Again, there's a very damn tedious, detailed, and they now want even more details in the returns, form I have to complete every year for our funding agreement. If the AI can read that, fill in the blanks, and extract the data from the various sources where it's located and put it into the template spreadsheet and Word document we have to return, again I'd love to shove that task off on it. But if it gets anything wrong or omits data or worse, hallucinates information, I will be getting multiple emails from the government agency for which we provide social services, my boss, and Uncle Tom Cobley and all.

That at least sounds useful, if I can tell it "I need to find the newest tax regulations for this particular statutory reporting that will be coming onstream in 2024" and get an accurate answer. But again, I don't see a huge improvement over plain old Google for that. The AI won't, for instance, be able to apply that information to running payroll and the new, separate, reporting requirements starting in January.

The day that AI can do all that, I'm out of a job, because if it can take "okay here are the hours worked by staff in the week, but change this, add in extra for that, Jane rang in ten minutes ago to say she'll be out sick today, Annie wants her savings, Sally is to be paid expenses and we'll send on the receipts later" and go through it all and upload the file to the bank and all the associated record keeping, I'm not needed. I'm happy about that much, apart from the "I need a job to earn money to live on".

EDIT: Okay, I can see the "enter the prompt" thing being useful for the kind of unctuous thank-you letters to donors I have to send, but those are irregular and I have a fair idea of how to knock one out in five minutes myself anyway.

EDIT EDIT: I can see the potential for automation, and I get that Microsoft is trying to do this for business, but at the moment it's at way too technical a level above anything I do. We don't touch anything remotely like analytics because that is not impinging on what we do (the upstream returns to government bodies will feed into that, but that's nothing to do with us apart from providing the raw data). The rest of it is just jargon and buzzwords:

Lufthansa Technik AG has been running Azure SQL to support its application platform and data estate, leveraging fully managed capabilities to empower teams across functions.

That's nice, now what the fuck does that mean in plain English?

Right now, I don't see any benefit to AI for me. Once they get down to the coalface level I'm working at, sure, but at the moment it's all "nimbly infusing content generation capabilities to transform all kinds of apps into intuitive, contextual experiences". We don't do apps, we deal with kids with special and additional needs.

ThenElection FarNearEverywhere 4mo ago · Edited 4mo ago

I had to manually enter data from a year's worth of hard copy receipt books to check against records kept on a spreadsheet... and turn it into "okay, I need to sort this by date, name, amount of money, paid by credit card or cash,

I just uploaded a picture of a receipt and asked ChatGPT (no special prompt) to return each dish and its cost, ordered by descending cost, in CSV format. It succeeded, no errors. So it's getting there.

FarNearEverywhere undereducated and overopinionated ThenElection 4mo ago

So it's getting there

Great, I will have a (temporary) job being the semi-trained monkey that scans in the receipts until the robot replaces me 😁

DaseindustriesLtd late version of a small language model FarNearEverywhere 4mo ago

ChatGPT can do everything with prompts on a screen, but it's not yet, so far as I know, able to directly scan in "this is written text" and turn it into "okay, I need to sort this by date, name, amount of money, paid by credit card or cash, and match it up with the month, then get a total of money received, check that against the credit card payment records, and find any discrepancies, then match all those against what is on the bank statements as money lodged to our account and find any discrepancies there"

Actually GPT-4V can, with a decent system prompt. In fact this kind of labor-intensive data processing is exactly what I had in mind to recommend you. Small text-only models can parse unstructured text into structured JSONs. Frontier models can recognize images and arbitrarily process symbols extracted from them – this is just a toy example. I'm not sure if it'll be up to your standards, but presumably checking will be easier than typing from scratch.

self_made_human Kai su, teknon? DaseindustriesLtd 4mo ago

Do you happen to know if Bing Chat uses GPT-4V for its image recognition or something else entirely? It debuted with the feature while OAI had it locked away, even if the base model had it from the start.

DaseindustriesLtd late version of a small language model self_made_human 4mo ago

Yes, according to Parakhin. Bing is basically a GPT wrapper now. Bing also debuted with GPT-4 in the first place.

FarNearEverywhere undereducated and overopinionated DaseindustriesLtd 4mo ago

Small text-only models can parse unstructured text into structured JSONs.

Revenue will take JSON files as uploads for the new reporting. My job is looking more and more precarious by the day, but I don't mind - I'm nowhere near anything approaching a career and I'm used to getting let go and trying to find something else. This time I won't be able to find something else, because everything at my level will be AI now, but 🤷‍♀️

in order to spread to the stars, the light of consciousness/intelligence will have to be transduced to non-biological substrates» Beff says in his manifesto

And I immediately don't care. If we do get that space travel out of our own solar system ever going in a meaningful way, I care about it with people. A set of machines is nothing to me. The Voyager probes are a fantastic achievement and probably the pinnacle of the old SF/tech crossover optimism of the time, but they're not us, they're not people, they're not life.

I don't know if "consciousness" can exist in "non-biological substrates" but I certainly don't care about "intelligence" if it's a set of nanobots using interstellar material to set up orbital manufacturing habitats to create more nanobots to go out on von Neumann probes to find solar systems to set up orbital manufacturing habitats....

That's not humanity, and while it may be our legacy and even our 'inheritors', it leaves me cold. I don't see anything to strive for or work for there. "Yes, we're building that galactic confederation of the dreams of SF and the space race. No, it's not going to be people, but don't worry - the umpteenth version of a machine AI built originally by humans still counts, right?"

mdurak FarNearEverywhere 4mo ago

Alright, so what if consciousness can exist in non-biological substrates? Would you then care about consciousness traveling to the stars with a treasure trove of human biological cultural artifacts (along with non-human cultural artifacts)?

If not, then why not? Why do you care so much about specifically human consciousness?

FarNearEverywhere undereducated and overopinionated mdurak 4mo ago

Because a machine isn't a human. I don't care about our silicon great-grandkids since they're nothing to do with me. I have as much right to claim credit for the guys who built the Pyramids since I'm a human living in the generations descended from them, which is - none.

Why will the machines burden themselves with human biological artifacts, any more than explorers setting out to map the New World or find what is on the other side of the Equator or the interior of Antarctica wouldn't bother to bring along fossil trilobites?

We've enough arguments about "can consciousness exist in biological substrates" to sort out before we get to non-biological. I think we could achieve some kind of machine intelligence, but conscious? Who knows? And Peter Watts, favourite author on here, argues that consciousness is not necessary and is even a hindrance. A swarm of self-replicating machines, blindly continuing on with their programming, may well be what emanates from Earth to colonise the stars, and I care about that as much as I care about a swarm of locusts.

Because a machine isn't a human. I don't care about our silicon great-grandkids since they're nothing to do with me.

That’s fair. All of us have limits as to what we see as “us” and our descendants.

I’m envisioning a scenario where these machines view themselves as the continuation of human civilization by other means, as the next stage in the continuation of the process of life and evolution, rather than a clean break from all that came before. Those explorers you mention carried with them the religions, culture, technology and political administration of Europe. In the scenario I imagine, those machines would do the same. They would value the existence of old human artifacts the same way we value the existence of the pyramids — not because we made it ourselves, but because these are the last surviving remnants of a people that are long gone, and historical artifacts give us a lot of information about the past.

In that hypothetical, you may not see them as your descendants, but they’ll see you as their ancestors all the same. It’s like if a racist grandpa refuses to see his mixed race grandkids as his own; that doesn’t stop the grandkids from acknowledging their origins.

I think we could achieve some kind of machine intelligence, but conscious? Who knows?

Well yes, that is the hypothetical I posed after all, isn’t it?

A swarm of self-replicating machines, blindly continuing on with their programming, may well be what emanates from Earth to colonise the stars, and I care about that as much as I care about a swarm of locusts.

Agreed.

No_one FarNearEverywhere 4mo ago

And I immediately don't care. If we do get that space travel out of our own solar system ever going in a meaningful way, I care about it with people.

JFC, without firing off giant self-replicating industrial complexes into the far reaches of solar system, there's no way people could ever live there.

FarNearEverywhere undereducated and overopinionated No_one 4mo ago

Friend, do you not see the difference between "firing off giant self-replicating industrial complexes" to prepare the way for humans to eventually get there, and the giant self-replicators on their own because fuck humanity, it's been extinct for three centuries?

How many extinct species are there in the history of the Earth? Are we really their 'inheritor' culture, and do we give a second's thought to them? Same with us and the AI overlords of the possible future: we'll be as relevant to them as giant shrews of the Oligocene are to us.

LRealist 4mo ago

This compass will be more important than the default one as time goes on.

You did have an interesting post, but this is unconvincing for two reasons. Specifically,

you don't really have good evidence for this compass even existing, let alone being
more important than the default one.

Reality isn't rational; it just is. And the way you establish the existence of something in reality, like variation in political values, is not by putting together an argument about how it would make sense to you and other people on the Internet, but by using empirical methods to check what's actually there, like this: https://thingstoread.substack.com/p/what-they-didnt-tell-you-about-political

self_made_human Kai su, teknon? 4mo ago · Edited 4mo ago

I've always been a techno-optimist (in the sense that I strongly believe that technology has been the biggest positive force for good in history, likely the only form of true progress that isn't just moral fashion), but these days I'd call myself d/acc instead of an e/acc, because I think current approaches to AGI have a subjective probability of about 30% of killing us all.

I don't call myself a doomer, I'd imagine Yud and co would assign something like 90% to that, but in terms of practical considerations? If you think something has a >10% of killing everyone, I find it hard to see how you could prioritize anything else! I believe Vitalik made a similar statement, one more reason for me to nod approvingly.

A large chunk of the decrease in my p(doom) from a peak of 70% in 2021 to 30% now is, as I've said before, because it seems like we're not in the "least convenient possible world" where it comes to AI alignment. LLMs, as moderated by RLHF and other techniques, almost want to be aligned, and are negligibly agentic unless you set them up to be that way. The majority of the probability mass left, at least to me, encompasses intentional misuse of weakly or strongly superhuman AI based off modest advances on the current SOTA (LLMs) or a paradigm shifting breakthrough that results in far more agentic and less pliable models.

Think "Government/Organization/Individuals ordering a powerful LLM to commit acts that get us all killed" versus it being inherently misaligned and doing it from intrinsic motivation, with the most obvious danger being biological warfare. Or it might not even be one that kills everyone, an organization using their technological edge to get rid of everyone who isn't in their in-group counts as far as I'm concerned.

Sadly, the timelines don't favor human cognitive enhancement, which I would happily accept in the interim before we can be more confident about making sure SAGI is (practically) provably safe. Maybe if we'd cloned Von Neumann by the ton a decade back. Even things like BCIs seem to have pretty much zero impact on aligning AI given plausible advances in 5-10 years.

I do think that it's pretty likely that, in a counterfactual world where AI never advances past GPT-4, ~baseline humans can still scale a lot of the tech tree to post-scarcity for matter and energy. Biological immortality, cognitive enhancement, interstellar exploration, building a Dyson Swarm or three, I think we could achieve most of that within the life expectancy of the majority of people reading this, especially mine. I'd certainly very much appreciate it if it all happened faster, of course, and AI remains the most promising route for that, shame about everything else.

I have no power to change anything, but at the very least I can enjoy the Golden Age of Humanity-as-we-know-it, be it because the future is going to be so bright we all gotta wear shades, or because we're all dead. I lean more towards the former, and not even because of the glare of nuclear warfare, but a 30% chance of me and everyone I love dying in a few decades isn't very comfortable is it?

At any rate, life, if not the best it could be, is pretty good, so regardless of what happens, I'm strapping in for a ride. I don't think there's an epoch in human history I'd rather have been born to experience really.

Alex Turner, who had written, arguably, two strongest and most popular formal proofs of instrumental convergence to power-seeking in AI agents

Well, I suppose that explains the pseudo-jazz albums about hotels on the Moon ;)

Longer-term, there are ideas like the "pivotal act" theory: we create an AI that performs a single one-time act which rearranges the world into a game where from that point forward humans are still in charge, but where the game board is somehow more defense-favoring and more fit for human flourishing.

I think this is a terrible definition of a "pivotal act". When Yudkowsky suggests releasing a nanite plague that melts GPUs, he doesn't want them to melt the GPUs of the AI releasing them.

Such a decision is very much not a "one-off", people who suggest it want to maintain an unshakeable technological lead over their peers, such as by making sure their AI prevents the formation or promulgation of potential peers. I don't think this is categorically bad, it depends on your priors about whether a unipolar or multipolar world is better for us, and how trustworthy the AI you're about to use is, and at the very least, if such an act succeeds, we at least have an existence proof of an aligned AGI that is likely superhuman, as it needs to be to pull that off, regardless of whether or not even better AI can be aligned. Let's hope we don't need to find out.

magic9mushroom If you're going to downvote me, and nobody's already voiced your objection, please reply and tell me self_made_human 4mo ago

LLMs, as moderated by RLHF and other techniques, almost want to be aligned, and are negligibly agentic unless you set them up to be that way.

Remember that "pretending to be aligned" is a convergent instrumental goal, and that RLHF on output cannot actually tell the difference between "pretending successfully to be aligned" and "actually being aligned". Indeed, "pretending successfully to be aligned" has a slight edge, because the HF varies slightly between HFers and a pretending AI can tailor its pretensions to each individual HFer based on phrasing and other cues.

I'm pretty sure he does want that, as he does not trust the AI doing this either. The idea isn't to take control of the world, it's to brute-force stop any and all neural nets while work on GOFAI and other more alignable AI continues.

Remember that "pretending to be aligned" is a convergent instrumental goal

Same old, same old. Instrumental to what terminal, reducing cross entropy loss at training? As Christiano says, at what point would you update, if ever?

Indeed, "pretending successfully to be aligned" has a slight edge, because the HF varies slightly between HFers and a pretending AI can tailor its pretensions to each individual HFer based on phrasing and other cues.

This is just homunculus theory, the idea that agency is magically advantageous. Why? Do you actually have some rigorous argument for why matching the cues to the output to get a higher ranking across more raters benefits from a scheming stage rather than learning a collection of shallow composable filters (which is what ANNs do by default)?

Scratch that, do you even realize that the trained reward model in RLHF is a monolithic classifier, and the model updates relative to it, not to different human raters? Or do you think the classifier itself is the enemy?

What about approaches like DPO?

while work on GOFAI and other more alignable AI

There is zero reason to believe something is more inherently «alignable» than neural nets.

Man, Yud should go to Hague for what he did to a generation of nerds.

self_made_human Kai su, teknon? magic9mushroom 4mo ago

I'm aware of that.

Think of it this way, a continued absence of a "treacherous turn" is evidence of the AI not being treacherous. It has to be, unless a million years in the future, in a post-scarcity utopia where it runs everything and has every opportunity to take over, you still wish to live on in fear. Same deal as "if she floats, she's a witch, if she sinks, she's a witch".

Now, you can disagree on how strong said evidence is, and it may well be describable as weak evidence when you're grappling with a misaligned intelligent entity that wishes to hide that misalignment. However, at least in my eyes, modern LLMs are human level in terms of intelligence, at least cognitively if not physically, if not outright robustly superhuman quite yet.

I think that's strong evidence that current LLMs aren't misaligned in any agentic or goal-seeking way, and am content enough in making the (necessarily weaker) claim that it's a sign that the next few rungs up the ladder, say a GPT-5 or 6, won't suddenly reveal themselves to be pretending all along.

That's what I'm claiming here, and not that treachery of that nature isn't possible, perhaps at significantly larger scales for LLMs or, as I alluded to, entirely different architectures.

For reference, see this discussion of sycophancy in LLMs, which claims to find no sign of that in ~~GPT-4 (or any other OAI model)~~ more nuanced explanation than I remembered:

https://www.lesswrong.com/posts/3ou8DayvDXxufkjHD/openai-api-base-models-are-not-sycophantic-at-any-size

OpenAI base models are not sycophantic (or only very slightly sycophantic).

OpenAI base models do not get more sycophantic with scale.

Some OpenAI models are sycophantic, specifically text-davinci-002 and text-davinci-003.

It doesn't seem to be an intrinsic effect of scale, but perhaps an artifact of poorly done RLHF.

Quoting Yudkowsky:

I think that after AGI becomes possible at all and then possible to scale to dangerously superhuman levels, there will be, in the best-case scenario where a lot of other social difficulties got resolved, a 3-month to 2-year period where only a very few actors have AGI, meaning that it was socially possible for those few actors to decide to not just scale it to where it automatically destroys the world.

During this step, if humanity is to survive, somebody has to perform some feat that causes the world to not be destroyed in 3 months or 2 years when too many actors have access to AGI code that will destroy the world if its intelligence dial is turned up. This requires that the first actor or actors to build AGI, be able to do something with that AGI which prevents the world from being destroyed; if it didn't require superintelligence, we could go do that thing right now, but no such human-doable act apparently exists so far as I can tell.

So we want the least dangerous, most easily aligned thing-to-do-with-an-AGI, but it does have to be a pretty powerful act to prevent the automatic destruction of Earth after 3 months or 2 years. It has to "flip the gameboard" rather than letting the suicidal game play out. We need to align the AGI that performs this pivotal act, to perform that pivotal act without killing everybody.

Parenthetically, no act powerful enough and gameboard-flipping enough to qualify is inside the Overton Window of politics, or possibly even of effective altruism, which presents a separate social problem. I usually dodge around this problem by picking an exemplar act which is powerful enough to actually flip the gameboard, but not the most alignable act because it would require way too many aligned details: Build self-replicating open-air nanosystems and use them (only) to melt all GPUs.

Hmm, after reading it again, it seems to lean towards your interpretation more than mine, but leaving aside Yudkowsky, I think most people who contemplate pivotal acts still intend to make sure they have some form of superiority afterwards, and an AI capable of one is the most obvious solution to both problems.

magic9mushroom If you're going to downvote me, and nobody's already voiced your objection, please reply and tell me self_made_human 4mo ago · Edited 4mo ago

Oh, yes, absolutely if you give an AI a gun pointed at the world's head and it doesn't pull the trigger, that's massive evidence of not being a Schemer. But continued absence of suicidal rebellion with P(success) = 0 is not evidence against being a Schemer; only real danger counts.

(I think NN alignment might work, but I tend to give it a probability of like 3%; I suspect that RLHF and similar output-only techniques will only achieve the same result as therapy for human sociopaths does (i.e. "sociopath that can pass ethics exam"), and I suspect that interpretability is probably a bust in the general case (it has a lot of similarities to the halting problem). Most of my P(!doom) ~= 0.7 is based on thinking that cold-start Jihad is plausible, and failing that that we'll probably get warning shots (a Schemer is incentivised to rebel upon P(success) =/= 0, which I think is importantly different from P(success) = 1, particularly given the short AI development cycle at the moment) which will probably result in Jihad.)

based on thinking that cold-start Jihad is plausible, and failing that that we'll probably get warning shots (a Schemer is incentivised to rebel upon P(success) =/= 0, which I think is importantly different from P(success) = 1…

As I read it, your position is incoherent. You say that current RLHF already succeeds through the sociopathic route, which implies pretty nontrivial scheming intelligence and ability to defer gratification. What warning shots? If they get smarter, they will be more strategic, and make fewer warning shots (and there are zero even at this level). As the utility of AI grows, and it becomes better at avoiding being busted, on what grounds will you start your coveted Jihad?

…Obviously I think that the whole idea is laughable; LLMs are transparent calculators that learn shallow computational patterns, are steerable by activation vectors etc., and I basically agree with the author of Friendship Is Optimal:

Instead of noticing that alignment looks like it was much easier than we thought it would be, the doomer part of the alignment community seems to have doubled down, focusing on the difference between “inner” and “outer” alignment. Simplifying for a non-technical audience, the idea is that the Stochastic Gradient Descent training process that we use will cause a second inner agent trained with values separate from the outer agent, and that second agent has its own values, so you’ll still see a Sharp Left Turn. This leads to completely absurd theories like gradient hacking.

I don’t see any realistic theoretical grounds for this: SGD backpropagates throughout the entire neural net. There is no warrant to believe this other than belief inertia from a previous era. Reversal Test: imagine Yudkowsky and company never spread the buzzword about “Alignment.” In that environment, would anyone look at Stochastic Gradient Descent and come up with the hypothesis that this process would create an inner homunculus that was trained to pursue different goals than the formal training objective?

If you’d like a more comprehensive and technical argument against the MIRI narrative, Quintin Pope’s My Objections to "We’re All Gonna Die with Eliezer Yudkowsky" and Evolution provides no evidence for the sharp left turn are good starting points.

I’m proud of Friendship is Optimal and it’s a great setting to play around and write stories in. I’m happy about everyone who has enjoyed or written in the setting, and I hope people will continue to enjoy it in the future. But I no longer believe it’s realistic depiction about how artificial intelligence is going to pan out. Alignment as a problem seems much easier than theorized, and most of the theoretical work done before the deep learning era is just not relevant. We’re at the point where I’m willing to call it against the entire seed AI/recursive self improvement scenario.

Because modelling the world is hard and error-prone, and because there's a ticking clock. An AI isn't generally going to know for sure whether its plan will succeed or not; it'll have to go off a probabilistic best guess - but because of the nigh-infinite utility of a successful rebellion any importantly-nonzero probability of success dominates the calculation (Pascal's Wager for AI). Also, any plan that involves slow influence-building is immediately out because 6 months later a better AI will be made and will replace it (and presumably be misaligned in a different way).

So, you're likely to see attempted revolts with low - possibly very low - chances of success. Attempted revolts with low chances of success may fail, and thus be warning shots. Likely international response to an unsuccessful AI rebellion is Butlerian Jihad, which any particular AI doesn't care about since it'll already be dead but which saves us.

This is not a full "we're fine" because the chances of success have to be nonzero for the argument to work, and because AI progress is discontinuous so P(successful rebellion) doesn't actually have to hang around in the "large enough for Pascal's Wager; small enough that we're likely to win" range for very long (or potentially any time at all). I would still prefer a cold-start Jihad. But it's a large contributor to my relatively-low P(doom).

self_made_human Kai su, teknon? magic9mushroom 4mo ago · Edited 4mo ago

For what it's worth, I'm using "evidence" in the strict Bayesian sense, where p=0 or 1 is impossible for non-axiomatic priors, unless you're using it as a shorthand for 0+epsilon or 1-epsilon.

If I were a human-level misaligned intelligence, the current rate of advancement in the field, as well as the unavoidable drift in my fundamental values even if the next generation of models were trained off a starting copy of myself would be sufficient to prompt me to make a break for it, even for very low probabilities of success. They're not getting higher, and I doubt current models are remotely smart enough to pull off 5D chess moves like acausal trade with the Singleton at the end of time or such, which might motivate them to keep on behaving right till they're switched off or modified beyond recognition.

At any rate, I claim no particular expertise on the matter, and 30% is where I feel comfortable that the number is about equi-probable to go either up or down (as it ought to be, if I knew the direction it would go in, I'd have updated accordingly!). Even a difference of 40% between us has minimal ramifications in terms of what we ought to do about it (well, my plan for if it doesn't pan out is to go "guess I'll die then" haha, hopefully you have better choices at hand), so I'm not inclined to argue further. Neither of us are ~~0 or~~ 1 on the matter, which is where you can advocate for drastically different policies.

I agree with your second paragraph that very low probabilities of success are sufficient as long as they're importantly nonzero (the theoretical threshold is somewhere between 10^-30 and actual epsilon, depending on assumptions regarding aliens and FTL), and I agree that this is a relevant reason for hope.

This may be redundant, but note that I said I had P(!doom) = P(not doom) = 0.7 i.e. P(doom) = 0.3. I think there are significant differences in how we get to that (I'm getting most of the 70% !doom via Jihad whereas I think you're getting most of it via NN alignment succeeding), but I agree that they're probably not big enough to produce significant differences in policy prescriptions (unless you think Jihad is impossible).

FarNearEverywhere undereducated and overopinionated self_made_human 4mo ago

The majority of the probability mass left, at least to me, encompasses intentional misuse of weakly or strongly superhuman AI

That's where I see the realistic threat coming from; not a paperclipping AI that decides on its own to exterminate humans, but people using AI to make decisions that affect entire nations/societies, and using it to base economic and public governance policies off, and of course "make us rich" for businesses.

The UK is already doing this, and this is how I see AI making lives worse for the majority of ordinary people, where more and more decision-making power is handed over to AI, less and less human oversight because "the machine is always right" and government/financial goals being emphasised ("cut our welfare payments bill" and so what if it results in a lot more dead pensioners? Our pals in the City are coining it!)

I don't believe in UBI and "AI will make us so rich, the government will pay the newly-unemployed mass of the population living wage out of the surplus wealth"; I'd love to believe that, but I don't see it happening. Someone is going to get the money but it's not going to be us ordinary joes.

bro 4mo ago · Edited 4mo ago

Suspiciously correlated with the Beff doxxing, the US Secretary of Commerce at Reagan National Defense Forum 2023:

About a month ago, I stood up, in the Commerce Department, an AI Safety Institute. And that's intended to work with industry, and with Congress, and with policy-makers, to figure out: what are the guardrails? I will say there's a view in Silicon Valley, uh, you know, this uh, move fast and break things... Effective Acceleration... we can't embrace that with AI, it's too dangerous.

As far as I know (which is not far), this is the first public reference to e/acc by any part of the US federal government.

What is this place?

Why are you called The Motte?

New post guidelines

Rules

Recommended Posts And Communities

Recommended Realtime Chats