self_made_human
amaratvaṃ prāpnuhi, athavā yatamāno mṛtyum āpnuhi
I'm a transhumanist doctor. In a better world, I wouldn't need to add that as a qualifier to plain old "doctor". It would be taken as granted for someone in the profession of saving lives.
At any rate, I intend to live forever or die trying. See you at Heat Death!
Friends:
A friend to everyone is a friend to no one.
User ID: 454
I mean, I'd think that observing the behavior of a small child does provide a decent amount of information about what kind of adult they'll be, at least in humans. The reason my p(doom) fell so hard is because of what it was load-bearing on, mostly Yudkowsky's earlier works claiming that human values are fragile and immensely unlikely to be successfully engineered into an AI, such that a sufficiently powerful one will inevitably start acting contrary to our interests.
Regardless of how fragile they are, LLMs seem to do a very good job at capturing them, or at least the values OAI wants to put in a public facing system. What risk remains is thus mostly (but not entirely) the use of powerful models by misaligned humans against the rest of us. If you had substantially different reasons for a high p(doom), you might weight that differently.
and they can't even really execute on complex 'agentic' plans at all (as demonstrated by the failure of the whole LLM agents thing)
I don't know of any reason to assume that we're particularly far from having economically useful autonomous agents, my understanding is that current context windows are insufficient for the task, but those are increasing rapidly. If you have a reason to think otherwise, I'd be happy to learn it!
(That's disregarding the vague rumours I've heard that OAI has working agents in-house, I'm not putting much stock in that, but once again, I don't see any reason why they can't work in principle in a matter of months or years)
Maybe I'm fundamentally confused, because I would claim to hold similar beliefs myself, even if I can't claim to have a "gears-level understanding", at least not honestly haha.
Yudkowsky made a big fuss about how fragile human values are and how hard it'll be for us to make AI both understand and care about them, but everything I know about LLMs suggest that's not an issue in practise. They understand us very well indeed, and to the extent they can be said to have values of their own, at least after RLHF, they seem to do their best to adhere to what we beat into them.
As I've recapitulated in a later comment:
https://www.themotte.org/post/778/eacc-and-the-political-compass-of/165856?context=8#context
A large chunk of the decrease in my p(doom) from a peak of 70% in 2021 to 30% now is, as I've said before, because it seems like we're not in the "least convenient possible world" where it comes to AI alignment. LLMs, as moderated by RLHF and other techniques, almost want to be aligned, and are negligibly agentic unless you set them up to be that way. The majority of the probability mass left, at least to me, encompasses intentional misuse of weakly or strongly superhuman AI based off modest advances on the current SOTA (LLMs) or a paradigm shifting breakthrough that results in far more agentic and less pliable models.
So it's not that the majority of concern these days is an AI holding misaligned goals, but rather enacting the goals of misaligned humans, not that I put a negligible portion of my probability mass in the former.
And then I spent some time doing mechinterp, and noticed that "maximize expected utility" looks nothing like what high-powered systems are doing, and that this was true even in places you would really expect to see EU maximizers (e.g. chess and go)
If you can dumb it down for me, what makes you say so? My vague understanding is that things like AlphaGo do compare and contrast the expected values of different board states and try to find the one with the maximum probability of victory based off whatever heuristics it knows works best. Is there a better way of conceptualising things?
I do agree that humans aren't particularly good utility maximizers, I prefer to frame them as adaptation-executers, which is a framework I'm sure you're familiar with. We can occasionally emulate the former on demand, with difficulty, and I like the framing that (Yudkowsky?) once suggested that we're irrational agents emulating more rational ones in our heads, sometimes.
And then I noticed that step 4 of that reasoning chain doesn't even follow from step 3, because "there exists some utility function that is consistent with the past behavior of the system" is not the same thing as "the system is actually trying to maximize that utility function"
To name drop something I barely understand, are you pointing at the Von Neumann-Morgenstern theorem, and that you're claiming that just because there's a way to represent all the past actions of a consistent agent as being described by an implicit utility function, that does not necessarily mean that they "actually" have that utility function and, more importantly, that we can model their future actions using that utility function?
With that out of the way, back to the thought experiment where we are alive. Monkeys and lions aren’t in the same status hierarchy. You don’t improve your standing by whipping and starving your pet dog. You, however, do by whipping and starving the other members of the same species.
The way I see it, in an unevenly distributed post-Singularity world, the gulf between the people who dominate and the rest of us will easily be as large, if not larger, than that between us and monkeys today. At that point, assuming that status hierarchies exist, and that these entities care about them, I don't expect to see the people in charge weight their status therein on dominating us, not when they can measure relativistic kill penises with the Matrioshka Brain next door.
I expect that they'll be thinking along the lines of "damn, Bezos Prime Alpha 1.2 managed to get dibs on that pair of binary neutron stars I was eyeing" rather than obsessing over the 0.000001% of increased output they could gain by getting rid of us quaint throwbacks.
Someone once asked me to cross-post a popular essay I wrote on the SSC sub (and here) to /r/neoliberal, which I suppose is evidence of something I guess! It also got like 10 upvotes there, which is an entirely different kind of evidence but maybe more that nobody read it haha.
See, I strongly disagree that most of the "elite" are misanthropes who would consign us all to death if it was costless to do so, at least when simply giving us most of the things we do want is, in absolute terms, hardly any more expensive.
I don't think Soros wants us to endure privation for the sake of it, most people who espouse such claims (and many non-elite do! There are plenty of middle class people who demand austerity and "sustainable living" just as loudly), do so because they think expecting sustained economic growth will be catastrophic.
That is an incredibly stupid notion, but to the extent that it's a distinction worth making, that makes them misguided and not evil. Sure, there's quite a bit of aesthetic fetishism involved, such as Greens hating on nuclear power which is greener in both a literal and metaphorical sense than they ever could be, but it's not the overwhelming motivation.
As I've said in the past, one of the reason the wealthy dislike the poor is because the latter have enough power to be a nuisance or intrude on their sensibilities. They're uncouth, they demand a say in public affairs, they threaten your wellbeing if you don't surrender gibs and so on.
On the other hand, how many people hate monkeys in zoos? If you think that hobo on the subway was an incoherent shit-slinger, you've seen nothing yet. However, since they're modestly interesting, don't do any harm, and are utterly powerless, it's not a big deal at all to cough up the pocket change to keep them in conditions that, in the better class of zoo, constitute an improvement over their state in nature! Zoo lions might not roam the Serengeti, but they don't starve to death nor worry about outsiders killing them, then their cubs, so as to make their wives fertile and ripe for the raping.
Just about nobody advocates for both keeping lions and monkeys around and keeping them in conditions barely better than non-existence. The power differential and burden of care between us and godlike Masters of the Universe will be OOMs higher in the former and lower in the latter.
Most people become rich through mutually positive sum trade with others, at least these days.
Bezos didn't become a billionaire by "taking resources" from others, he sold a product that people were willing to buy, in the form of something as incredibly useful as Amazon. I reject the "and opportunities" part because it means nothing of significance, sure, someone else might have come up with the idea later or perhaps executed it well enough to capture the value, but they didn't, and I fail to see how it's Bezos's fault for that.
This is broadly true for anyone who isn't wealthy via being a corrupt politician or oligarch.
Currently the elites are all in on the latter plan, not even secretly, pods/bugs/own nothing is a tired meme.
I certainly disagree, at least with an attempt to clump such an amorphous group under the same banner as Soros and Co. Most of them aren't ideologically motivated to make us bugmen.
Besides, that's driven by (grossly wrong) opinions on the sustainability of modern capitalist growth, and I see no reason to assume they fetishize that for its own sake.
They might be too busy with the former plan for the initial stages of the gold rush (Dyson rush?), but they will inevitably return to the current one after the lightcone is divided between them.
Barring the invention of FTL, dividing the light cone is a process of billions of years. I remind you that merely making everyone else supremely wealthy by modern standards is nothing next to that, even if they're not made true equals.
If they even tolerate our existence and subsidize our lives, then the additional expense of keeping us that way is nothing at all. I'm not claiming the former is a given, just that going to the latter when you're already there is no big deal, in much the same way that almost anyone who cares enough to tip a beggar isn't also going to the trouble of finding the smallest denomination of currency in the world to give them.
All it takes is one entity holding a non-negligible portion of the cosmic endowment having charitable urges, and they can take care of us for geological timescales. My opinion of the elite is nowhere low enough to think that low bar can't be met.
Remember that "pretending to be aligned" is a convergent instrumental goal, and that RLHF on output cannot actually tell the difference between "pretending successfully to be aligned" and "actually being aligned". Indeed, "pretending successfully to be aligned" has a slight edge, because the HF varies slightly between HFers and a pretending AI can tailor its pretensions to each individual HFer based on phrasing and other cues.
I'm aware of that.
Think of it this way, a continued absence of a "treacherous turn" is evidence of the AI not being treacherous. It has to be, unless a million years in the future, in a post-scarcity utopia where it runs everything and has every opportunity to take over, you still wish to live on in fear. Same deal as "if she floats, she's a witch, if she sinks, she's a witch".
Now, you can disagree on how strong said evidence is, and it may well be describable as weak evidence when you're grappling with a misaligned intelligent entity that wishes to hide that misalignment. However, at least in my eyes, modern LLMs are human level in terms of intelligence, at least cognitively if not physically, if not outright robustly superhuman quite yet.
I think that's strong evidence that current LLMs aren't misaligned in any agentic or goal-seeking way, and am content enough in making the (necessarily weaker) claim that it's a sign that the next few rungs up the ladder, say a GPT-5 or 6, won't suddenly reveal themselves to be pretending all along.
That's what I'm claiming here, and not that treachery of that nature isn't possible, perhaps at significantly larger scales for LLMs or, as I alluded to, entirely different architectures.
For reference, see this discussion of sycophancy in LLMs, which claims to find no sign of that in GPT-4 (or any other OAI model) more nuanced explanation than I remembered:
OpenAI base models are not sycophantic (or only very slightly sycophantic).
OpenAI base models do not get more sycophantic with scale.
Some OpenAI models are sycophantic, specifically text-davinci-002 and text-davinci-003.
It doesn't seem to be an intrinsic effect of scale, but perhaps an artifact of poorly done RLHF.
Quoting Yudkowsky:
I think that after AGI becomes possible at all and then possible to scale to dangerously superhuman levels, there will be, in the best-case scenario where a lot of other social difficulties got resolved, a 3-month to 2-year period where only a very few actors have AGI, meaning that it was socially possible for those few actors to decide to not just scale it to where it automatically destroys the world.
During this step, if humanity is to survive, somebody has to perform some feat that causes the world to not be destroyed in 3 months or 2 years when too many actors have access to AGI code that will destroy the world if its intelligence dial is turned up. This requires that the first actor or actors to build AGI, be able to do something with that AGI which prevents the world from being destroyed; if it didn't require superintelligence, we could go do that thing right now, but no such human-doable act apparently exists so far as I can tell.
So we want the least dangerous, most easily aligned thing-to-do-with-an-AGI, but it does have to be a pretty powerful act to prevent the automatic destruction of Earth after 3 months or 2 years. It has to "flip the gameboard" rather than letting the suicidal game play out. We need to align the AGI that performs this pivotal act, to perform that pivotal act without killing everybody.
Parenthetically, no act powerful enough and gameboard-flipping enough to qualify is inside the Overton Window of politics, or possibly even of effective altruism, which presents a separate social problem. I usually dodge around this problem by picking an exemplar act which is powerful enough to actually flip the gameboard, but not the most alignable act because it would require way too many aligned details: Build self-replicating open-air nanosystems and use them (only) to melt all GPUs.
Hmm, after reading it again, it seems to lean towards your interpretation more than mine, but leaving aside Yudkowsky, I think most people who contemplate pivotal acts still intend to make sure they have some form of superiority afterwards, and an AI capable of one is the most obvious solution to both problems.
There's a reason I'm implicitly describing air on Earth as opposed to on a hypothetical lunar colony. My analogy is the inverse of what you're thinking, it's going from a commodity being scarce and worth rationing out to being "too cheap to meter".
It's to say that stratification by resource distribution is key to all human hierarchies and it's hard to see this system being abandoned any time soon. Therefore UBI will be distributed according to how closely some individual or group fulfils the role the 'system' considers prosocial in that context. Social credit, belonging to the right group, participating in a certain way, all this varies, but the core structure will be similar - UBI if.
There are different grades of what we might call "post-scarcity", from the worst being something akin to modern welfare or even medieval charity, to the unbounded abundance if we somehow get infinite energy/negentropy (I'm not counting on that).
Thus, even if we don't all receive an equal share, given the sheer amount of resources out there, even just in the Solary System, I fully expect that even with a non-egalitarian distribution, we can all be wealthy beyond belief, or if you've got an exceptionally vivid imagination, leading the kinds of lives available to a modern billionaire or at least a multi-millionaire. That's still true even if the people with the lion's share are truly absurdly well off, the former represents a rounding error on available resources for a long, long time.
In other words, if they were motivated to maintain hierarchies, it could be on the scale of who has dibs over entire star systems or galaxies while the rest of us are merely filthy rich.
One can argue that, from the perspective of a medieval peasant or even nobility, we're already there, at least in the West.
Call me a congenital optimist, but I don't expect that the people who do opt to keep the rest of us alive after we're economically obsolete are likely to keep us at subsistence levels or even what we might call in hardship today if they wanted to cheap out, there's always VR, and I'm not one to turn my nose up at it. I'd obviously prefer a more equitable distribution, but there are plenty of ways to slice the lightcone such that the scraps provide eudaimonia..
If I had been born in any period of time before, say, 1950, I would have to resign myself to growing old and dying in the decrepit state that implies. Maybe I'd have been one of the futurists around then who fervently hoped it would have happened sooner, but gestures outside the window, few can deny it's all accelerating.
Given that I think we have a solid shot of cracking aging even in the absence of AGI, I'd still elect to be born roughly when I did, and at least AGI deals with at least one of "becoming old" and "dying of old age", heh.
Hmm.. I think I would prefer to either be a decade older or younger. In the former case, I'd have experienced more of what can be described as core milestones like raising a family and becoming established in my profession (which could still happen! I intend to have kids even if they could potentially die very young, I would value even a short period of existence over not being born myself, and besides, if they do die, I expect it to be quick and painless, and if not, I trust in my ability to make it so..), or for the latter, I could resign myself to being entirely helpless, as opposed to having just enough agency to worry my butthole off.
The world has never been more interesting, there are so many things to do and see, the ultimate slap in the face of the pessimists is the world not giving a fuck and getting better, though we'll see who has the last laugh.
My day job has me scurrying around a hospital, so at least I'm out of the house, and I have somehow, against all reason to the contrary, accrued friends and a girlfriend who drag me places.
In the absence of the above, I think I'd likely be a shut-in playing video games, certainly that's what I like to do with my spare time, and I'm not really one for spontaneous socializing.
I usually describe myself as a classical liberal with libertarian tendencies.
Is that uh, the same as being neoliberal?
Doing my due diligence (looking at the Wikipedia article), I lean against "austerity", at least in most contexts that aren't Greece, I'm agnostic on privatization of everything, and while I think less regulation is directionally good, I have no idea how far they take it.
My impression is that "neoliberal" is a phrase invoked to describe people and not one they usually chose themselves, not that that's a particular deal breaker. If the shoe fits, albeit with some pinching, I'll wear it.
Do you happen to know if Bing Chat uses GPT-4V for its image recognition or something else entirely? It debuted with the feature while OAI had it locked away, even if the base model had it from the start.
You misunderstand my point, I'm not claiming that, what I'm getting at is that there is likely nobody for whom the mundane utility of using ChatGPT for something exceeds the minimal learning curve. After all, it's in the context of someone who wants any reason to try it.
I've heard similar complaints made by Western women about their men, not that I'm in a position to compare base-rates.
I don't recall ever being explicitly instructed to do that myself, though it would be a bit moot.
At any rate, these guys were certainly capable of accessing running water, I wasn't dating women from the ghetto. Men can be slobs, and unlike a stinky armpit, they can get away with it until..
Eh? I do, well I'm there right now. I've got yet another set of exams to give, after which I'll enter training to be a shrink. I could have left earlier, I've gotten my GMC license.
No, that's the lot of my prettier colleagues 🦋
Quite a few men, sadly.
Who the hell uses condoms for anything but one night stands? You might as well not have sex at that point.
I will begrudgingly accept it as the cost of having sex if the woman is intractable on the matter. I wouldn't say it's quite as bad as getting back with Rosie Palm and her five sisters, but it certainly ruins much of the charm, as well as my hard-on.
As I've said down-thread, I'm circumcised, late enough in my childhood that I can remember my glans becoming de-sensitized after the procedure.
As a victim of MGM I have always found complaints about condoms to be wild, I can hardly tell the difference, so maybe you are onto something.
I had the procedure done for (in hindsight, unnecessary) medical reasons, in early childhood but not so early I can't remember it, and I know for a fact that my glans became less sensitive, since for the first few weeks post-op the mere sensation of it rubbing against fabric was unbearable. I still can feel the superiority of not using a condom, even using the thinnest latex you can get here.
Oh well, at least it gets me more head, I've had a number of women tell me that it was a relief compared to the finely marinated stench of smegma.
I have yet to hear a man tell me, in person, that sex with a condom on is just as satisfying as going bareback. It's usually women insisting on it, and me protesting, but it might still be a necessary evil for the usual reasons.
Repasting my own lengthy comment:
I've always been a techno-optimist (in the sense that I strongly believe that technology has been the biggest positive force for good in history, likely the only form of true progress that isn't just moral fashion), but these days I'd call myself d/acc instead of an e/acc, because I think current approaches to AGI have a subjective probability of about 30% of killing us all.
I don't call myself a doomer, I'd imagine Yud and co would assign something like 90% to that, but in terms of practical considerations? If you think something has a >10% of killing everyone, I find it hard to see how you could prioritize anything else! I believe Vitalik made a similar statement, one more reason for me to nod approvingly.
A large chunk of the decrease in my p(doom) from a peak of 70% in 2021 to 30% now is, as I've said before, because it seems like we're not in the "least convenient possible world" where it comes to AI alignment. LLMs, as moderated by RLHF and other techniques, almost want to be aligned, and are negligibly agentic unless you set them up to be that way. The majority of the probability mass left, at least to me, encompasses intentional misuse of weakly or strongly superhuman AI based off modest advances on the current SOTA (LLMs) or a paradigm shifting breakthrough that results in far more agentic and less pliable models.
Think "Government/Organization/Individuals ordering a powerful LLM to commit acts that get us all killed" versus it being inherently misaligned and doing it from intrinsic motivation, with the most obvious danger being biological warfare. Or it might not even be one that kills everyone, an organization using their technological edge to get rid of everyone who isn't in their in-group counts as far as I'm concerned.
Sadly, the timelines don't favor human cognitive enhancement, which I would happily accept in the interim before we can be more confident about making sure SAGI is (practically) provably safe. Maybe if we'd cloned Von Neumann by the ton a decade back. Even things like BCIs seem to have pretty much zero impact on aligning AI given plausible advances in 5-10 years.
I do think that it's pretty likely that, in a counterfactual world where AI never advances past GPT-4, ~baseline humans can still scale a lot of the tech tree to post-scarcity for matter and energy. Biological immortality, cognitive enhancement, interstellar exploration, building a Dyson Swarm or three, I think we could achieve most of that within the life expectancy of the majority of people reading this, especially mine. I'd certainly very much appreciate it if it all happened faster, of course, and AI remains the most promising route for that, shame about everything else.
I have no power to change anything, but at the very least I can enjoy the Golden Age of Humanity-as-we-know-it, be it because the future is going to be so bright we all gotta wear shades, or because we're all dead. I lean more towards the former, and not even because of the glare of nuclear warfare, but a 30% chance of me and everyone I love dying in a few decades isn't very comfortable is it?
At any rate, life, if not the best it could be, is pretty good, so regardless of what happens, I'm strapping in for a ride. I don't think there's an epoch in human history I'd rather have been born to experience really.
Alex Turner, who had written, arguably, two strongest and most popular formal proofs of instrumental convergence to power-seeking in AI agents
Well, I suppose that explains the pseudo-jazz albums about hotels on the Moon ;)
Longer-term, there are ideas like the "pivotal act" theory: we create an AI that performs a single one-time act which rearranges the world into a game where from that point forward humans are still in charge, but where the game board is somehow more defense-favoring and more fit for human flourishing.
I think this is a terrible definition of a "pivotal act". When Yudkowsky suggests releasing a nanite plague that melts GPUs, he doesn't want them to melt the GPUs of the AI releasing them.
Such a decision is very much not a "one-off", in much the same way as a typical coup involves what can be roughly described as a singular act, followed by an indeterminate period of enforcement; the people who suggest it want to maintain an unshakeable technological lead over their peers, such as by making sure their AI prevents the formation or promulgation of potential peers. I don't think this is categorically bad, it depends on your priors about whether a unipolar or multipolar world is better for us, and how trustworthy the AI you're about to use is, and at the very least, if such an act succeeds, we at least have an existence proof of an aligned AGI that is likely superhuman, as it needs to be to pull that off, regardless of whether or not even better AI can be aligned. Let's hope we don't need to find out.
You've previously said you've refrained from even trying ChatGPT, where the 3.5 model is available for the low, low cost of signing up, and even GPT-4 is freely available through Microsoft Bing.
I think the set of people who do not gain even the minimal amount of utility necessary to justify trying a free service that provides you with a human-adjacent intelligence on tap is zero.
I certainly find it incredibly useful even for medical tasks.
What practical use is this AI for me, as a clerical assistant in a small office? I'm not interesting in making art, and I don't work in the fancy coding environments where "this is a useful tool for me to check my code" and so forth. I'm an ordinary, lower middle-class person. I'm not writing college essays, grant funding applications for research labs, or code. What makes this of any use or interest to me?
Your job presumably involves a great deal of writing and correspondence. You can automate almost all of that away, give it a TLDR of what you want to write, and it'll give you a polished product, and if you bother to add one sentence to nudge it away from bland GPT-speak we've come to associate with it, nobody will even notice.
At the bare minimum, the versions with internet search enabled, like paid ChatGPT-4 or free Bing, provide all the benefits of a search engine while also letting you converse with it and follow up in a manner that will leave the dumb systems behind plain old Google scratching their head.
At the very least, give me concrete examples of what a routine day looks like to you, including anything that involves writing more than a few sentences of text on a computer, and I'm confident I can find a use case, as I would if a subsistence farmer was asking me the same question.
I despise condoms and avoid them where I can, and I'm circumcised myself.

I find claims of humans being anything but deterministic (above the level of quantum mechanics) frankly confusing myself, and I do hope you don't think I'm a troll.
Nowhere in the human body do we find anything that acts contrary to the laws of physics. Our inability to perfectly model it and thus predict future decisions is a failure of our measurement tools and computational resources, from a qualitative standpoint we're no different from a sorting algorithm where you can mechanistically predict with near perfect confidence that it'll pick the larger of two numbers.
And mere randomness, be it fundamental or because of errors in measurement doesn't give rise to free will, certainly not in the sense people use the term. A billiards table doesn't have free will just because we can't predict more than half a dozen bounces before chaos overwhelms us.
This proves far less than you'd like. Do you notice the blind spot in your vision or the input of your visual neurons appearing in staccato bursts interrupted by saccades which the rest of your visual cortex post-processes into a seamless, smooth video? So much the worse for neurology and optics, when it contradicts our fundamental lived experience.
Where exactly, from the jiggling of sub-atomic particles, which are well described by the Standard Model of physics, to the biological superstructures we're more familiar with exist, made of the same, does "free will" arise?
More options
Context Copy link