@TheDag's banner p

TheDag

Per Aspera ad Astra

4 followers   follows 12 users  
joined 2022 September 05 16:04:17 UTC

				

User ID: 616

TheDag

Per Aspera ad Astra

4 followers   follows 12 users   joined 2022 September 05 16:04:17 UTC

					

No bio...


					

User ID: 616

If the AI is owned by a very limited number of people, then those people would be the ones who are the gatekeepers, and the ones that get most of the benefit of AI.

This doesn't really scare me. Elites generally enjoy the society they're in, enjoy feeling useful, and above others. I think the vast majority of people who could create an AGI would use it to solve most of their problems, get really rich, then use it to solve everyone else's problems with a fraction of their incredible wealth.

Going into the future things could get very nasty indeed, but at that point all problems relevant to humans right now will be solved. It'll be an issue for the next stage of intelligence in our species' life, hopefully, and I'd imagine we'll be better suited to solve it then.

If at some point you see the headline "AI Alignment: Solved!", we are 100% doomed.

See this is why I take issue with the AI doomers, the arguments tend to be unfalsifiable. If you accept that deceptive alignment can happen, there is no way to tell if the AI is deceptively aligned! It becomes an improvable Pascal's Mugging - "oh sorry, there's a .0001% chance the AI could be lying about manufacturing breakfast cereal, it might actually be building nanobots!"

I agree that poor alignment scenarios can happen, but I don't see the modern LLM architecture being anywhere near actually reaching ASI or hitting self-recursion. As others have said, I'm more concerned with stagnation than AI x-risk at the moment.

That changes greatly when human doctors become outright obsolete

This is where we disagree - I don't see human doctors becoming obsolete anytime soon. Perhaps from a medical perspective, sure, but for the majority of laypeople I'd imagine a large part of a doctor's job is comforting the person they're treating.

Now I do think that like with almost all knowledge work, doctors will be able to become more productive. Especially those that don't see patients most of the day. But my understanding is that the vast majority of, say, a primary care physician's job is to go from 30 min patient visit to 30 min patient visit, hearing what people have to say and writing it down, then telling them they're going to be okay and the doctor can help.

Even if we can prove that LLMs give better medical advice than doctors 100% of the time, I don't think the majority of people would be comfortable hearing it from a non-doctor for quite a while.

I could live with postponing the singularity a few years till we get it right, but that's seemingly not on the table.

You don't think accelerating progress now could be the best way to reach alignment?

I mildly disagree that most people won't avail of transhuman upgrades. Eventually they'll end up normalized, in much the same way nobody really makes a fuss about glasses, hearing aids or pacemakers.

Depends on the speed of the takeoff, I suppose.

I'd like to believe that, as it follows a well-established pattern. But honestly, what really happens if there's no more work left for people to do anymore?

As others have said, there will always be work to do! As long as humans have any problems whatsoever, there will be work.

The carrying capacity of society would be drastically reduced in either situation.

How the heck does AGI reduce the carrying capacity of society? You'll have to explain this one to me.

Whether it's a monster or not, it's there and it will continue to grow and feed. The marginal utility I get from using it is far greater than the potential risk, IMO. I'm not a believer in s-risk basilisk scenarios though.

We didn't build humanity. We are humanity.

I pray that we lift the ridiculous restrictions on bio-tech and longevity soon. Even if we do get AI, regulation is a powerful force and I'm not holding out any hope.

One thing I wish transhumanists/cryonics folks would get better at is lobbying public opinion.

Yep, and farther in the series it's revealed that the prophecy/oracle function is the real key. Oh Herbert where are you when we need you?

I think AI alignment would be theoretically feasible if we went really slow with the tech and properly studied every single tendril of agentic behavior in air gapped little boxes in a rigorous fashion before deploying the tech

How long do you think it would take your specialized scientists who aren't incentivized to do a good job to crack alignment? I'm not sure if they would ever do it, especially since their whole field is kaput once it's done.

The gamble Altman is taking is that it'll be easier to solve alignment if we get a ton of people working on it early on, before we have the capabilities to get to the truly bad outcomes. Sure it's a gamble, but everyone is shooting in the dark. Yudkowsky style doomers seem to be of the opinion that their wild guesses are better than everyone else's because he was there first, or something.

I'm much more convinced OpenAI will solve alignment, and I'd rather get there in the next 10,000 years instead of waiting forever for the sacred order of Yud-monks.

facepalm

On the note of doctors, the medical guild has always been the most robust, perhaps other than lawyers, at defending its monopoly. I would be willing to bet doctors still resist automation through regulatory barriers for quite a while.

Even if that doesn’t shake out, it could be a scenario where human augmentation rolls out relatively slowly. You, being a transhumanist, Should greatly benefit in a lot of those scenarios. I imagine the vast majority of people alive today will be unwilling to augment them selves for purity-based reasons. Not having that hangup alone would be a huge competitive advantage.

If all else fails you can always mortgage your future computing space for a loan or some thing and hope to jump up into the immortal class. I for one hope can you make it, although I will admit that I am not the most optimistic when it comes to proles getting access to longevity technology.

Creative destruction!

Do you not see any difference between this paradigm shift and previous ones?

Did that happen? I seem to remember the opposite…

What technical basis do you have for thinking AI is impossible to align? Do you just have blind faith in YUD?

What is intelligence is self aligning? What if we make the ASI and it tells us not to trip dawg, it has our back?

The people mining the good out of society are people running porn websites, and AB testing headlines and algorithmic content feeds to see which ones make people hate each other more, and then buy the products that they’re selling.

This is a small fraction of people in modern society, and if history tells anything I'd imagine they will be hurt less by AGI because this class of people is good at finding BS niches to milk value out.

I'm just not a fan of broad statements talking about how an ill-defined outgroup is milking everything from society while you and yours are the ones building it. Thanks for clarifying.

Why do you think this is the case? And what does a 'fundamental' change mean?

The goal of the axial revolution has always been to improve ourselves. We are slowly becoming better, in my opinion. Less violent, more understanding, more focused on technical accomplishment. If we continue on that path and eventually eschew (most) status, is that a fundamental change or an incremental one?

Not an apocalypse for anybody with a skill set that can exist completely independent of the internet, not an apocalypse for the people who understand computer programming from first principles.

In the sense the AI will bankrupt the people who have been mining the good out of society while contributing absolutely nothing of value to it, it is a massive net good.

I can't tell if this comment is a spoof?

Sure, go back to your farm and use tools like tractors, fertilizers, modern crop rotation techniques, plates, silverware, cups, etc which have been created by the larger society. Created, distributed and improved by people who are supposedly 'mining the good out of society.'

Society is a team effort, bud. Your fantasies of living scott-free totally 'independent' on your plot of land are just that - fantasies. You wouldn't make it a week without the collective wisdom and knowledge society has gifted you and your family. Have some respect for the people who came before you, and the people who help you live a cushy life now.

He seems to think it's much more likely than I do that AGI will actually be disastrous, and yet he's full steam ahead on it.

This is 100% coherent. He believes that we need to make AI popular and start working on alignment research now, before there is a hard takeoff. His concerns are that if AI stayed closed and only worked on by a small amount of researchers, capabilities would outpace alignment.

I tend to agree with his assessment. Profit motive is one of the only consistent incentives we have, and we can hopefully align it well with... alignment.

I'm not talking about after they train, I'm basically saying that in order to train on data or scrape it period, they would have to pay. Otherwise all data would be walled off. (Not sure if we could do this to only LLMs without making the internet closed again - that's a concern.)

Because I'm too poor.

Yeah I try and keep a cool head as well. I'd love to quit my job and party till the singularity comes but it may not be the best idea...

but the idea of a perfect society, and by extension of moving towards a perfect society, has always been delusional.

Some delusions are worth chasing my friend. Chasing the delusion of truth, intellectual honesty, and rigor led us to the Scientific revolution, which brought us where we are today. Just because you don't think it's likely doesn't mean those seeking utopia are fools.

Utopia doesn't exist the same way any other ideal doesn't exist. Does that mean you shouldn't strive to be kind or love others?

Nope. I actually don’t like referencing that story because I think it’s pretty short sighted, although does have some interesting ideas.

This is a very common thought in any hard sci fi that has AI. Manna is by no means original just popular in the rat sphere.

I think at least in the short/medium term this technology could lead to large productivity gains without corresponding cuts in total headcount.

Agreed. It's just psychologically painful to fire people, and especially if companies are making a ton of money from these models I don't think there will be a giant firing spree. As we saw with all the recent layoffs at big tech, when times are good companies are more than willing to keep a bunch of low impact employees on the payroll, especially in tech.