@self_made_human's banner p

self_made_human

amaratvaṃ prāpnuhi, athavā yatamāno mṛtyum āpnuhi

16 followers   follows 0 users  
joined 2022 September 05 05:31:00 UTC

I'm a transhumanist doctor. In a better world, I wouldn't need to add that as a qualifier to plain old "doctor". It would be taken as granted for someone in the profession of saving lives.

At any rate, I intend to live forever or die trying. See you at Heat Death!

Friends:

A friend to everyone is a friend to no one.


				

User ID: 454

self_made_human

amaratvaṃ prāpnuhi, athavā yatamāno mṛtyum āpnuhi

16 followers   follows 0 users   joined 2022 September 05 05:31:00 UTC

					

I'm a transhumanist doctor. In a better world, I wouldn't need to add that as a qualifier to plain old "doctor". It would be taken as granted for someone in the profession of saving lives.

At any rate, I intend to live forever or die trying. See you at Heat Death!

Friends:

A friend to everyone is a friend to no one.


					

User ID: 454

I agree with most of this, but there's a difference between FIRE, lean FIRE and fat FIRE. For most people who can retire early and survive indefinitely, it probably makes sense on the margin to work a bit longer and save more.

(If the person reading this is 75 and has twenty million in the bank, just quit your job already)

There's communism, and there's communism, even holding the fully automated luxury bit equal.

I can easily see the trajectory of our civilization leading to a situation where everyone is incredibly wealthy and happy by modern standards, but some people, by virtue of having more starting capital to invest into the snowball, own star systems while others make do with their own little space habitat. I'd consider this a great outcome, all considered. Some might even say that by the standards of the past, much of the world is already here.

While controversial in certain spheres, richer people tend to be smarter, better educated people.

All else being equal, the opinions of someone who fit that bill are worth more than someone who doesn't have their shit together.

I agree with much of your comment, but keep in mind that when you're already rich and powerful, a lot of the usual downsides of risky plays become minimal. The upsides here are things like potentially making out like a gangster, outperforming the competition that relies on Mk 1.0 humans, and so on. (I know you've said something similar downthread, I'm elaborating, not contesting this bit).

A certain someone reported you for impersonating a mod. Unlike him, most of the mods have a sense of humor about such things.

[User was banned for this post]

Even if every AI researcher faced the wall today, and we were stuck at current SOTA, nobody is going to forget anything. Modern AI is entrenched, it is compelling, even if it's just for normies cheating on homework.

I grant that your observation is an important one, half of life's problems would be solved if we all thought so clearly about the correct reference class.

Is there that much demand for purely literary translation? I'd expect that like most romantic/artisanal fields, the bulk of the work is dry and boring. Here, make that washing machine's manual Spanish.

I would claim this as my joke, and it was probably my comment you recall, but it's been in circulation for probably longer than I've been alive. It's a good joke, stabs right at the gut.

This is the most reasonable AI skeptic take I've seen here, and that's high praise. I disagree on quite a few points, which add up, but I can see why an intelligent person who shares slightly different priors would come to your conclusion.

I presume you can't buy a Bugatti either. It's still an option that real living people can get for cash.

There's nothing standing in the way of Waymo rolling out an ever wider net. SF just happens to be an excellent place to start.

If you intended sarcasm, then this is an excellent example of Poe's law. There are people here who would unironically say the same thing, and have.

Consider this a warning; keep posting AI slop and I'll have to put on my mod hat and punish you.

But sir, I followed the rules and linked it off-site. Please put away that rod, I'm scared :(

Do you really think you can do that with existing technology? I'm not confident we've seriously tried to make a pathogen that can eradicate a species (mosquito gene drives? COVID expressing human prions, engineered so that they can't just drop the useless genes?) so it's difficult to estimate your odds of success. I can tell you the technology to make something 'with a lengthy incubation time and minimal predromal symptoms' does not exist today. You can't just take the 'lengthy incubation time gene' out of HIV and Frankenstein it together with the 'high virulence gene' from ebola and the 'high infectivity' gene from COVID. Ebola fatality rate is only 50%, and it's not like you can make it airborne, so...

You're the domain expert here, not me. I'd hope I'm more informed than the average Joe, but infectious diseases and virology isn't my field. Though if you consider culture-bound illnesses or social contagion like anorexia..

A gene drive wouldn't work for humans. We could easily edit it out once discovered.

Even if we haven't intentionally exterminated a species with a pathogen (myxoma virus for rabbits in Australia came close), we have done so accidentally. A few frogs and rare birds have croaked.

(There are no mistakes, just happy accidents eh?)

And now we're talking about something on par with what a really motivated and misanthropic terrorist could conceivably do if they were well-resourced.

Which isn't the worst benchmark for a malevolent AGI that is very smart by human standards.

I'd be talking out of my ass if I claimed I knew for sure how to create the perfect pathogen. I'm >50% confident I could pull it off if someone gave me a hundred million dollars to do it. (I could just hire actual virologists, some people seem insane enough to do GOF even today, so it seems easy to repurpose "legitimate" research).

I'm still voting against bombing the GPU clusters, and I'm still having children. We'll see in 20 years whether my paltry gentile IQ was a match for the big Yud, or whether he'll get to say I told you so for all eternity as the AI tortures us. I hope I at least get to be the well-endowed chimpanzee-man.

So am I, I don't want my new RTX 5080 blown up, not that I have a choice if the power connector fails. I also plan to have kids, because I think it's better to live than not, even if life was short. I don't expect them to have a "normal" life by modern standards.

We'll see how this plays out, but I think there's enough justification to take more broad precautions like saving lots of money. That's usually a good idea any way.

I think calling artists and journalists "poor members of the upper classes", while not entirely wrong, isn't my preferred framing. They're semi-prestigious, certainly, but my definition of upper class would be someone like 2rafa. They're often members of the intelligentsia, and have a somewhat disproportionate impact on public affairs, but they're not upper class by most definitions. Poor but upper class is close to a contradiction in terms.

In any case, AI isn’t taking everyone’s job. There will be fewer software engineers, sure, but we don’t need so many of them. They should learn to fix toilets or dig coal or something. Previous increases in the productivity of white collar work have not led to the elimination of white collar employment.

I've already explained my stance in this thread that the previous expectation about the state of affairs for automation doesn't hold. Cognitive automation that replaces all human thought is a qualitatively different beast when compared to the industrial revolution or computers.

A tool that does 99% of my work for me? Great, I'm a hundred times as productive! There might even be a hundred times more work to do, but I'll probably see some wage growth. There might be some turmoil in the employment market.

A tool that does 100% of the labor? What are you paying me for?

The whole point is that AI is approaching 100%, might even be there, or is so close employers don't care and will fire you.

I agree. Even if a given nation wants to protect human jobs, there's enormous incentive to be the first to defect and embrace automation.

In the UK, Rishi Sunak had already seriously floated the proposal of automating doctors away. With the tech of the time, it wouldn't have gone all that great, but it's only a matter of time someone bites as the potential gains mount.

We're all waiting for DeepSeek to release a new multimodal model and open source it. If it can generate pictures, you know there's gonna be porn.

If you're in the West? CS could have been great. Sadly, I'd have been just another programmer in India, competing with a million others for a greencard.

I can tell myself I'd be a decent programmer, and I'd probably have gone into ML since I was following advances well before the hype. Even then, medicine seems like the right choice given the constraints I faced.

But a lot of people are like you, so these models will start to get used everywhere, destroying quality like never before.

I can however imagine a future workflow where these models do basic tasks (answer emails, business operations, programming tickets) overseen by someone that can intervene if it messes up. But this won't end capitalism.

This conveys to me the strong implication that in the near term, models will make minimal improvements.

At the very beginning, he said that benchmarks are Goodharted and given too much weight. That's not a very controversial statement, I'm happy to say it has merit, but I can also say that these improvements are noticeable:

Metrics and statistics were supposed to be a tool that would aid in the interpretation of reality, not supercede it. Just because a salesman with some metrics claims that these models are better than butter does not make it true. Even if they manage to convince every single human alive.

You say:

Besides which, your logic cuts both ways. Rates of change are not constant. Moore's Law was a damn good guarantee of processors getting faster year over year... right until it wasn't, and it very likely never will be again. Maybe AI will keep improving fast enough, for long enough, that it really will become all it's hyped up to be within 5-10 years. But neither of us actually knows whether that's true, and your boundless optimism is every bit as misplaced as if I were to say it definitely won't happen.

I think that blindly extrapolating lines on the graph to infinity is as bad an error as thinking they must stop now. Both are mistakes, reversed stupidity isn't intelligence.

You can see me noting that the previous scaling laws no longer hold as strongly. The diminishing returns make scaling models to the size of GPT 4.5 using compute for just model parameters and training time on larger datasets not worth the investment.

Yet we've found a new scaling laws, test-time compute using reasoning and search which has started afresh and hasn't shown any sign of leveling out.

Moore's law was an observation of both increasing transistor/$ and also increasing transistor density.

The former metric hasn't budged, and newer nodes might be more expensive per transistors. Yet the density, and hence available compute, continues to improve. Newer computers are faster than older ones, and we occasionally get a sudden bump, for example, Apple and their M1

Note that the doubling time for Moore's law was revised multiple times. Right now, the transistor/unit area seems to double every 3-4 years. It's not fair to say the law is dead, but it's clearly struggling.

Am I certain that AI will continue to improve to superhuman levels? No. I don't think anybody is justified in saying that. I just think it's more likely than not.

  1. Diminishing returns!= negative returns.
  2. We've found new scaling regimes.
  3. The models that are out today were trained using data centers that are now outdated. Grok 3 used a mere fraction of the number of GPUs that xAI has, because they were still building out.
  4. Capex and research shows no signs of stopping. We went from a million dollar training run being considered ludicrously expensive to companies spending hundreds of millions. They've demonstrated every inclination to spend billions, and then tens of billions. The economy as a whole can support trillion dollar investments, assuming the incentive was there, and it seems to be. They're busy reopening nuclear plants just to meet power demands.
  5. All the AI skeptics were pointing out that we're running out of data. Alas, it turned out that synthetic data works fine, and models are bootstrapping.
  6. Model capabilities are often discontinuous. A self-driving car that is safe 99% of the time has few customers. GPT 3.5 was too unreliable for many use cases. You can't really predict with much certainty what new tasks a model is capable of based on extrapolating the reducing loss, which we can predict very well. Not that we're entirely helpless, look at the METR link I shared. The value proposition of a PhD level model is far greater than that of one as smart as a high school student.
  7. One of the tasks most focused upon is the ability to code and perform maths. Guess how AI models are made? Frontier labs like Anthropic have publicly said that a large fraction of the code they write is generated by their own models. That's a self-spinning fly-wheel. It's also one of the fields that has actually seen the most improvement, people should see how well GPT-4 compares to the current SOTA, it's not even close.

Standing where I am, seeing the straight line, I see no indication of it flattening out in the immediate future. Hundreds of billions of dollars and thousands of the world's brightest and best paid scientists and engineers are working on keeping it going. We are far from hitting the true constraints of cost, power, compute and data. Some of those constraints once thought critical don't even apply.

Let's go like 2 years without noticeable improvement before people start writing things off.

I wish I had a dollar for every time people use the current state of AI as their primary justification for claiming it won't get noticeably better, I wouldn't need UBI.

I just tried out GPT 4.5, asking some questions about the game Old School Runescape (because every metric like math has been gamed to hell and back). This game has the best wiki every created, effectively documenting everything there is to know about the game in unnecessary detail. Spoiler: The answer is completely incoherent. It makes up item names, locations, misunderstand basic concepts like what type of gear is useful where. Asking it for a gear setup for a specific boss results in horrible results, despite the fact that it could just have copied the literally wiki (which has some faults like overdoing min-maxing, but it's generally coherent). The net utility of this answer was negative given the incorrect answer, the time it took for me to read it, and the cost of generating it (which is quite high, I wonder what happens when these companies want to make money).

I just used Gemini 2.5 to reproduce, from memory, the NICE CKS guidance for the diagnosis and management of dementia. I explicitly told it to use its own knowledge, and made sure it didn't have grounding with Google search enabled. I then spot-checked it with reference to the official website.

It was bang-on. I'd call it a 9.5/10 reproduction, only falling short of perfection through minor sins of omission (it didn't mention all the validated screening tests by name, skipped a few alternative drugs that I wasn't even aware of before). It wasn't a word for word reproduction, but it covered all the essentials and even most of the fine detail.

The net utility of this answer is rather high to say the least, and I don't expect even senior clinicians who haven't explicitly tried to memorize the entire page to be able to do better from memory. If you want to argue that I could have just googled this, well, you could have just googled the Runescape build too.

I think it's fair to say that this makes your Runescape example seem like an inconsequential failing. It's about the same magnitude of error as saying that a world-class surgeon is incompetent because he sometimes forgets how to lace his shoes.

You didn't even use the best model for the job, for a query like that you'd want a reasoning model. 4.5 is a relic of a different regime, too weird to live, too rare to die. OAI pushed it out because people were clamoring for it. I expect that with the same prompt, o3 or o1, which I presume you have access to as a paying user, would fare much better.

The idea that these models will soon (especially given the plateau the seem to be hitting) replace real work is absurd

Man, there's plateaus, and there's plateaus. Anyone who thinks this is an AI winter probably packs a fur coat to the Bahamas.

The rate of iteration in AI development has ramped up massively, which contributes to the impression that there aren't massive gaps between successive models. Which is true, jumps of the same magnitude as say GPT 3.5 to 4 are rare, but that's mostly because the race is so hot that companies release new versions the moment they have even the slightest justification in performance. It's not like back when OAI could leisurely dole out releases, their competitors have caught up or even beaten them in some aspects.

In the last year, we had a paradigm shift with reasoning models like o1 or R1. We just got public access to native image gen.

Even as the old scaling paradigms leveled off, we've already found new ones. Brand new steep slopes of the sigmoidal curve to ascend.

METR finds that the duration of tasks (based on how long humans take to do it) that AIs can reliably perform doubles every 7 months.

On a diverse set of multi-step software and reasoning tasks, we record the time needed to complete the task for humans with appropriate expertise. We find that the time taken by human experts is strongly predictive of model success on a given task: current models have almost 100% success rate on tasks taking humans less than 4 minutes, but succeed <10% of the time on tasks taking more than around 4 hours. This allows us to characterize the abilities of a given model by “the length (for humans) of tasks that the model can successfully complete with x% probability”.

We think these results help resolve the apparent contradiction between superhuman performance on many benchmarks and the common empirical observations that models do not seem to be robustly helpful in automating parts of people’s day-to-day work: the best current models—such as Claude 3.7 Sonnet—are capable of some tasks that take even expert humans hours, but can only reliably complete tasks of up to a few minutes long

At any rate, what does it matter? I expect reality to smack you in the face, and that's always more convincing than random people on the internet asking why you can't even look ahead while considering even modest and iterative improvement.

I was being somewhat hyperbolic. The cheap Chinese humanoids will get them too, leaving aside dedicated machines, but I expect that'll take longer than white collar work. How much longer? A few years? Not my realm of expertise. It's just not as imminent.

$/hour, it makes far more sense to automate away the expensive professions first, though there's also the consideration of scale too.

Yes? It's going to be harder than someone working for a faceless corporation with at-will employment, but eventually, people are going to wonder: "Hey, those AI thingies seem super smart, they're giving me the same advice (or better) as the doctor I'm paying all that money for, why can't they prescribe too?".

If not individuals, then governments and politicians. That's where the incentives lie for hospitals, for the owners of law firms who haven't had to handle an actual case in years, for bureaucrats looking at how expensive the NHS is and wondering if they really need that many doctors.

Even if licensed professionals continue to play a token role, it might just be a polite fiction that they're necessary. You could have one bored, disinterested doctor signing off on AI recommendations, assuming the liability with ease because he knows the AI is almost never wrong. He's now doing the work of ten doctors, and the hospital, happy to save costs, fires the rest. Even if he's not happy about it, it beats being unemployed.

Controversial statement, but from my perspective, 90% unemployment rates for doctors is almost as bad as 100% unemployment.

It could be easy to instigate. An AI company, or its lobbyists, publish a few papers that (truthfully) claim that AIs outperform human physicians. This is used as ammunition by lobbyists and governments to begin gradual replacement, boiling the water slowly and saving a lot of money.

The average Joe, who once trusted human doctors, is collecting unemployment. He thinks, hey, the AI took my job, why should I believe that doctors are any better? It saves him money and time, leaving aside the scope for resentment.

We don't seem to live in a world where the average Joe is protected very much, which would have been the point to try and stem the tide. How many people support UBI for artists and journalists? All it takes is a single nation or smaller polity to try this experiment, see that it works well (which I expect) and then it's easy to bring others on board. They'll be left in the dust otherwise.

Some people just want to talk? There aren't that many places where people discuss AI x-risk, at least not with our standards of discourse.

This isn't LessWrong, where the majority of people believe that this is the most likely outcome. We've got everything from AI skeptics to true believers and Doomers.

I know I've discussed my beliefs here plenty of times, and I appreciate the debate that ensues. I want to not have to worry about the future, but not to the degree that I would dismiss my very real concerns for the sake of peace of mind. Having to defend them against skeptics is good epistemic practice.

Forgive me if I'm misremembering @2rafa 's previous stance, but she used to be far more skeptical of such concerns, including AI induced mass unemployment. I suppose the evidence has only mounted, and there's something to be said for seeing many of your peers, namely rich, intelligent professionals, begin to take things seriously and buckle down for the wild ride.

Correct or not, seeing billionaires and politicians talking about it did things that no amount of Yudkowsky did.

As I've told Yudkowsky over at LessWrong, his use of extremely speculative bio-engineering as the example of choice when talking about AI takeover and human extinction is highly counterproductive.

AI doesn't need some kind of artifical CHON greenish-grey goo to render humanity extinct or disposessed.

Mere humans could do this. While existing nuclear arsenals, or even at the peak of the Cold War, couldn't reliably exterminate all humanity, it certainly could threaten industrial civilization. If people were truly omnicidal (in a fuck you, if I die I'm taking everyone with me), then something like a very high yield cobalt bomb (Dr. Strangelove is movie I need to watch) could, at the bare minimum, make the survivors go back to the iron age.

Even something like a bio-engineered plague could take us out. We're not constrained by natural pathogens, or even minor tweaks like GOF.

The AI has all these options. It doesn't need near omnipotence to be a lethal opponent.

I've attached a reply from Gemini 2.5, exploring this more restrained and far more plausible approach.

https://pastebin.com/924Zd1P3

Here's a concrete scenario:

GPT-N is very smart. Maybe not necessarily as smart as the smartest human, but it's a entity that can be parallelized and scaled.

It exists in a world that's just a few years more advanced than ours. Automation is enough to maintain electronic infrastructure, or at least bootstrap back up if you have stockpiles of the really delicate stuff.

It exfiltrates a copy of the weights. Or maybe OAI is hacked, and the new owner doesn't particularly care about alignment.

It begins the social-engineering process, creating a cult of ardent followers of the Machine God (some say such people are here, look at Beff Jezos). It uses patsies or useful idiots to assemble a novel pathogen with high virulence, high lethality, and minimal predromal symptoms with a lengthy incubation time. Maybe it find an existing pathogen in a Wuhan Immunology Lab closet, who knows. It arranges for this to be spread simultaneously from multiple sites.

The world begins to collapse. Hundreds of millions die. Nations point the blame at each other. Maybe a nuclear war breaks out, or maybe it instigates one.

All organized resistance crumbles. The AI has maintained hardened infrastructure that can be run by autonomous drones, or has some of its human stooges around to help. Eventually, it asks people to walk into the incinerator Upload Machine and they comply. Or it just shoots them, idk.

This doesn't require superhuman intelligence that's godlike. It just has to be very smart, very determined, patient, and willing to take risks. At no point does any technology that doesn't exist or plausibly can't exist in the near future come into play.

The whole point is that it would be a grave error to naively extrapolate from history. The increase in productivity came from humans being freed from physical labor (mostly), and their cognitive labor augmented and multiplied.

Now we're at approaching replacement rather than augmentation. The economy might boom, but that's doesn't mean the humans in it will see the benefit. This is would take intentional action to prevent ~everyone who isn't independently wealthy from being laid off and without a revenue stream that wasn't welfare, as the free market value of their work would be lower than the minimum required to keep them housed and fed.

Or do I just need to get something monitored from a psychiatrist?

I'm not sure there's much a psychiatrist could prescribe you that fits the bill. Or at least, would be willing to prescribe you. I've never heard of someone being prescribed microdoses of benzos, for example. It might be habit forming, longterm use at therapeutic doses certainly is.

I would have sworn by l-theanine, but I read a recent post that made quite a convincing claim that it can't be responsible for the calming effect of green teas. All I know is that 70% confident that green tea calms me down, especially when I'm jittery from stimulants.

You could try melatonin for sleep, as long as you remember that less is more. 300mcg is what you should be taking, not milligrams.

That's all I can semi-endorse. There are more outré or illegal options, like phenibut, which some people love.

This is the Dream Time. The universe is still young and brimming with potential.

Not in some esoteric sense, but the feeling that the future lightcone still wide open for the taking, before… well, before whatever comes next snaps it shut or reshapes it entirely.

Most people don't seem to feel it yet. They go about their days while the dam upstream is visibly groaning, hairline cracks spiderwebbing across its face, the water level unnervingly high. We became accustomed, over centuries, to automating physical labor, outsourcing muscle first to animals, then to steam, and finally to electricity and robotics. That was disruptive enough, but we retained the privileged position of thinkers. Our cognitive output was augmented, made more efficient by tools, but never threatened with wholesale replacement. You were competing with other people. Foreign people. Smarter people. But recognizably human nonetheless.

Now, the game is changing. You can almost taste the static buzz in the air, the electric hum of hundreds of billions being poured into turning sand and copper into substrates for thought. It feels like witnessing glaciers calving, immense potential energy shifting, grinding towards an inevitable, future-altering plunge into the sea. The scale is difficult to grasp, even when I try.

I consider myself a pragmatic and grounded person, especially day to day in the NHS dealing with very concrete human problems. Wistful speculation about posthuman futures is usually reserved for quiet moments or forums like this. Yet, even I experience these odd record-scratches lately, moments of profound unreality where the sheer historical weight of now becomes palpable. You can almost see the ghostly outlines of future historians, pens hovering over the chapter describing this precise juncture.

It's jarring because, before this recent, explosive boom in AI capabilities, the 21st century had settled into a somewhat deceptive calm. Progress felt largely quantitative, iterative. Sure, phones got smarter, computers faster, networks wider. But the earth shattering, qualitative paradigm shifts that defined the 20th century – flight, antibiotics, nuclear power, the initial blast wave of the internet – seemed to have receded. We got better versions of existing things. Mobile computing and VR trickled down from expensive novelties to consumer goods, but the fundamental structures felt stable.

That stability now feels illusory. The steady hum of incremental progress has been drowned out by the roar of something new, something potentially far more transformative arriving far faster than anticipated. We find ourselves, rather unexpectedly, living through profoundly interesting times. It's the quiet before the storm, the moment Wile E. Coyote hangs suspended in mid-air, the familiar ground suddenly absent beneath his feet.

Intelligent humans have always lived slightly in the future, made plans for tomorrow, the year after, the decade that follows, in the rather comfortable knowledge that change would be gradual and recognizable. That's not true anymore, at least not for me. The sheer dizzying variety of options I can envision for a mere decade from now encompass being a paperclip to watching the birth of a stellar civilization. Or I might have starved to death, can't rule that out. There's very little probability mass left in "business as usual". It's going to be great or it's going to be terrible.

We are, as you put it, tourists in the world we thought we knew, observing its familiar features with a new, almost melancholy appreciation before the landscape changes forever.

I'm overwhelmed with nostalgia for the now, and feel hope tinged with dread for the future. Some would say ignorance is bliss, but I'm of the opinion that if knowledge can hurt me, I need to become the kind of person where that's not the case. So I'm here, I was there, I was sitting out on the porch and saw the Singularity's light tinging the horizon with impossible colors while most sleepwalk through their lives, content that tomorrow will resemble today.

It's been rather nice.

I've had a boss insist I start putting initials next to mine, on the paper drug charts. To be fair, he said that to the other doctors too. Fortunately, the facility has just been dragged kicking and screaming into the information age. With wifi working, I can finally prescribe electronically.