This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
You know what, I'm past caring. At least caring enough to make a fuss about things, or spend my limited time under the untamed sun trying to convince people to see the writing on the wall.
Yeah, we just found a solution to Navier-Stokes. I know just enough to say that that's a Big Deal. It's one of many Big Deals lately, each bigger than the last.
Anyone is welcome to plot the trendlines. I'm going to continue having fun and doing things IRL while that adds anything over what the bots can do. Good luck everyone, make sure to enjoy yourself.
As one of those people who, in your view, refuses to see the writing on the wall, I'm not sure why you would even be worried about my coming over to your side. Even if you could convince me and all the other AI skeptics out there, what, exactly, am I supposed to do with this information?
I am known to have altruistic tendencies. If that isn't enough, I dislike it when people are wrong about things, and find it satisfying to at least try and correct them. Also, there was some scope for coordinating a solution instead of having it imposed on us. Maybe it's not too late.
What are you going to do with this information? Entirely up to you. What would you do with the knowledge we're going to be raptured, or that an asteroid has a 30% chance of hitting the Earth in a few years?
Go to the pub and wait for it all to blow over?
That's what you would do if the asteroid was going to hit in a few hours, not a few years. We need some way to live while we wait, and with uncertainty about whether the asteroid is really going to hit at all.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I think in terms of psychological impact this kind of thing just isn’t important to most people. We’re going from a world where 99.9[99]% of people don’t contribute intellectually to the advancement of the human race to a world where 100% don’t. That affects the tiny minority who genuinely aspired to be in and believed they could be in the former group (even they were mostly destined for disappointment, AI or not), but not really everyone else.
Well, I suspect they'll care when they start ending up unemployed, or when the world reconfigures itself faster than they can blink. In the meantime, I'm sure ignorance is bliss.
That’s the thing though. As the other response says, ‘knowing’ (not really, but let’s say ‘having an idea of the scale of’) what’s coming doesn’t really help you in any way.
I disagree. Knowing that human endeavor is going to end up redundant in less than a decade has some obvious implications as far as I'm concerned.
I wouldn't quit my job to retrain in something else, the opportunity cost will never be recouped. I have to worry less about saving up for college for my kids - they never need to attend one for credentialing purposes. Any health related concerns that would take decades to kick in become far less pressing.
It should drastically change your priorities and medium and longterm planning. That's happened to me. But I prepare for as many scenarios as I can, and at least in my case, I don't need to do anything particularly out of the ordinary.
That being said, I should max out my ISA. I am not immune to procrastination.
What medium or long term planning is it changing for you? It’s not like a big drought or a tsunami or even a civilizational breakdown (or major war) where you can theoretically prepare if you have the means. I know people who are prepping, some at considerable cost, but they’re prepping for some kind of fantasy The Road type future which now seems unlikely. ASI will be far more powerful and wide reaching, its power will stretch beyond geography and the limits of our own civilization. Even the example you give (say, deciding not to commit to an expensive degree or career move) doesn’t necessarily make sense. If none of it matters, maybe you decide to do that masters in philosophy or take the job in some remote and low paid location anyway, just for fun.
As for human endeavor, I think things will be fine (absent x risk, which it now seems is impossible to control and so with which we are at the mercy of fate) if everything is automated. There will always be interesting things to do - think of the amateur woodworkers making flimsier furniture than IKEA, but having a great time doing so. Most of us will never be (close to) the best at anything, so ‘the best’ going from being a human to an AI doesn’t affect us in the same way that it does Terence Tao.
If we have a worst-case scenario, or just any of the really bad ones, we're fucked. A hidden bunker will just mean you die last.
If things go swimmingly, then the worrying would have been pointless - in hindsight.
That means I think it still makes sense to follow general, option-preserving strategies. That covers the increasingly unlikely "business as usual" worldlines, and shelters against socio-economic/political upheaval as the changes (good, bad or meh) promulgate.
In my context, that means I just continue being a doctor, progressing through training so I keep making more money. Money is the most general of strategies. It can be exchanged for goods and services, including more durable barter goods if shit really hits the fan.
I like money, you know. And want a family, which is expensive. I can't stress enough how the "correct" course of action depends on someone's priorities, which I am really not in a position to dictate. If you want to go be a hobo on a tropical beach, be my guest. My approach minimizes my hindsight regret.
I continue deriving some amount of satisfaction from my career, and regardless of how this ends, I'll say that I did something important while that meant something.
Surely that's the very best one can do, if "everyone dies" is what's happening?
(even leaving aside the possibility that it's actually almost everyone dies, ofc)
If.
It remains an open question, and I'm not as pessimistic as some.
More options
Context Copy link
More options
Context Copy link
I wasn’t arguing against the ‘business as usual’ strategy at all, for what that’s worth - after all, that’s what I’m doing.
But I was curious about the additional or different preparations you’ve made / steps you’ve taken that you referenced. The changes to plans.
I would have signed up for cryonics, before I was middle aged. Not much point in that now. I will reconsider only if things seem to peter out.
I have accelerated my long-standing goal of marrying and having kids. In a slower world, I'd have been okay with waiting a few extra years.
I should be putting all or most of my savings in an index fund. I've been procrastinating on that.
I am more open to risky medical treatments, to a degree, because the longterm damage is more likely to be fixable.
At a recent nadir of my depression, I reminded myself that I want to see how things end, or begin, really. That we'll find a cure for treatment resistant depression, if existing interventions fail. That I just needed to live long enough to see that through.
Well, I got better anyway. Mostly because of my own actions rather than anything any other shrink did for me. If that doesn't last, I can look forward to a technological fix in a very reasonable amount of time.
None of this is really life-changing. Which is something I've always noted when people argue that believing in an imminent techno-singularity requires drastic changes in behavior or outlook, therefore this discredits the belief in the first place. Beyond retarded as an argument, but also false when you consider I've been expecting this for a while, it's priced in, and I've had a pretty normal life anyway.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
How can one go from acknowledging enormity of the thing to being better prepared for being economically replaced by machines or rendered incapable of keeping track of accelerating reality?
I don't know. That's the honest answer.
How does one deal with the fact that they have a malignancy? That their loving grand uncle won the lottery, and might die in a few years and hand it to you?
Once you accept it as probable, your brain will reconfigure in the background. Maybe you can speed it up (I once did a heroic dose of psychedelics, but I wouldn't recommend that as a general solution. I wanted to get over depression and I was desperate), maybe you can't.
Maybe the knowledge consumes some people with unbearable anxiety. I'm sorry if that's the case. But I'm not in the business of reassuring lies, quite literally in terms of work. Either way, the facts are what they are.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Although this is clearly groundbreaking and historic and I’m not downplaying it in any way, at a glance this seems to be in line with the other major AI mathematics results we’ve seen lately, which mostly fall into the categories of either “noticing links between existing papers in the literature that no one had noticed before”, or “pulling a specific counterexample/construction out of the impossibly vast search space”. I don’t at all have the technical knowledge to evaluate how unexpected this result was, how close it is to existing methods/approaches, etc, but that’s what it looks like to my untrained eyes. I think we’re still waiting for that qualitative breakthrough where AI generates valuable new ideas ex nihilo that really change the way people think about a given domain; I think we’re still waiting for that qualitative breakthrough in math and in every other field. There’s a distinction between say, solving an individual problem and leaving it there, vs inventing something like Galois theory that spawns many new fundamental research directions. If a subject matter expert DOES think this paper will change the lay of the land in fluid dynamics (beyond just the significance of the result itself), please chime in.
It’s mostly a moot point though since all these questions are actually pretty easy to test empirically. If OpenAI and Anthropic are still employing humans, then there’s clearly still work to be done.
Imagine a car.
For a long time, it was a purely theoretical car. Spending any amount of time theorizing about its safety features was a charming activity for undersocialized nerds.
Then it moved. Slowly enough that you could walk alongside it. It was a mild curiosity, at best, to most.
Then it started picking up speed. The acceleration was increasing too, and some of us found that concerning. You could plot the bloody thing. I can't stress enough how little of this required a revelation.
I was around when we were trying to design safety features for a vehicle that didn't exist yet. Later, I was politely asking people to take notice when the ride started getting bumpy.
About a year and a half ago, I became somewhat frantic. Did people really not see what was coming? I spent far too much time arguing about it. I should have saved my breath.
Six months ago, with Mythos? The steering wheel came off. And I considered myself done with trying to convince anyone of anything.
I have the misfortune of occasionally reading someone explaining that the car can't be dangerous because of how the engine works. Or that it's a beat up sedan, and we should only worry about F1 cars. And don't you know that the speed of limit is a hard limit on velocity? Or that our current transmission systems break down at 570 km/h?
I'm sure there's a reply to that. I've probably written one before. If writing doesn't work, look out a window before it's all a blur.
Can people see an inch beyond their noses? Apparently not.
These days, I'm doing my best to strap myself in. I have no idea how much good that will do, but I've only got so much time and attention, and I'd like to spend some of it on the assumption that I have a future.
I don't know where the road ends. But some people are about to wish for airbags, and I find that a "told you so" is a small consolation.
More options
Context Copy link
That does sound like downplaying it, your preamble notwithstanding.
But, sure, we don't have an AI Grothendieck yet. It's been a full three hours since OpenAI announced a solution to Navier Stokes; but, what's it done for us lately, I think we've hit a wall, etc.
Give it a year, or six months, or three, who knows, and we'll be in a different place.
From what I’ve seen of the proofs, you still have a case of researcher + AI solving them rather than just pure AI. An “issue” with following the progress AI development is that until we reach AGI, which allows it to be independent and flip the board, you still will keep the paradigm of AI plus human even as the AI gets better and better. AI is going to keep getting more and more useful for a period of time until it becomes independent and we get full AGI. Until then the paradigm, and the existence of the researchers jobs even if the number is reduced, will not change.
More options
Context Copy link
It’s unfortunate that this whole discussion has been poisoned from the start because people can’t resist turning it into a competition. Granted, I completely understand why it’s deeply satisfying to run victory laps around the Gary Marcuses of the world and I don’t fault you for that.
OpenAI hasn’t sent everyone home yet and many people are still gainfully employed doing the sorts of things that people ordinarily do. Until that changes, I don’t see any problems with speculating about why that hasn’t happened yet, where the remaining gaps are, and what the nature of those gaps is.
More options
Context Copy link
He has a point. Most of the stuff I've heard of (I am not a mathematician) is proof by counterexample which doesn't per se expand the space of what we can do. I suspect the AI+researcher combo is still significantly more important than just AI for the stuff that actually transfers beyond mathematics.
Centaurs are unstable. Before long, AI+human combos are left behind by even stronger AIs, so strong that adding a human to the loop just makes it worse; the human slows the AI down, becoming a liability. In chess, that took ten years. How long will math take? Perhaps another year or two.
I know that this is what people say, but it's predicated on a scenario where the Venn diagram of AI capabilities (at a useable price even) becomes a strict superset of human capabilities. At the moment this hasn't happened even in software where AI cannot maintain a codebase long-term; studies showing that AI outcompetes centaurs are usually on short-term discrimination tasks like medical diagnoses.
I think it's quite likely that this scenario comes to pass, but it hasn't yet, so we'll see.
More options
Context Copy link
In some fields it's been this way for decades (hello air defense!); in others I suspect it will be a very long time before centaurs are replaced.
My sense is that even very smart LLMs are very gullible. I think it will be some time before centaurs aren't invaluable in conducting opposed work, which is the most important work of all. Pivoting models away from general purpose work towards specialized tasks may help here, even if that doesn't seem to be something the primes want to admit.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link