guajalote
11mo ago
The problem with the nanobot argument isn't that it's impossible. I'm convinced a sufficiently smart AI could build and deploy nanobots in the manner Yud proposes. The problem with the argument is that there's no need to invoke nanobots to explain why super intelligent AI is dangerous. Some number of people will hear "nanobots" and think "sci-fi nonsense." Rather than try to change their minds, it's much easier to just talk about the many mundane and already-extant threats (like nukes, gain of function bioweapons, etc.) that a smart AI could make use of.
guajalote
11mo ago
I meant "malignant" in the same sense as "malignant tumor." Wasn't trying to imply any deeper value judgment.
guajalote
11mo ago
If you're arguing about why AI will kill us all, yes, you need to establish that it is indeed going to be superhuman and alien to us in a way that will be hard to predict.
I don't even think you need to do this. Even if the AI is merely as smart and charismatic as an exceptionally smart and charismatic human, and even if the AI is perfectly aligned, it's still a significant danger.
Imagine the following scenario:
-
The AI is in the top 0.1% of human IQ.
-
The AI is in the top 0.1% of human persuasion/charisma.
-
The AI is perfectly aligned. It will do whatever its human "master" commands and will never do anything its human "master" wouldn't approve of.
-
A tin-pot dictator such as Kim Jong Un can afford enough computing hardware to run around 1000 instances of this AI.
An army of 1000 genius-slaves who can work 24/7 is already an extremely dangerous thing. It's enough brain power for a nuclear weapons program. It's enough for a bioweapons program. It's enough to run a campaign of trickery, blackmail, and hacking to obtain state secrets and kompromat from foreign officials. It's probably enough to launch a cyberwarfare campaign that would take down global financial systems. Maybe not quite sufficient to end the human race, but sufficient to hold the world hostage and threaten catastrophic consequences.
guajalote
11mo ago
That midjourney stuff is utter pabulum. It's only beautiful by the most shallow and insipid standards of beauty. The kind of "beauty" that would rank Thomas Kinkaide's paintings above Rembrandt's, because the former is bright and sparkly while the latter is brown and muddy. Or the kind of "beauty" that would consider N*SYNC's music superior to Bach's because the former's is free of dissonance and the later's is rife with it.
I don't particularly like the human art you linked either, but at least the artists are trying to do something interesting. We can do better than ugly modern art without resorting to saccharine crap and calling it beauty.
AI gives people what it gets positive feedback from. It gives people what they want.
Marvel movies and McDonalds chicken nuggets are examples of giving people what they want. Mass appeal produces boring hyperpalatability, not greatness.
guajalote
11mo ago
I'm not criticizing midjourney, I'm criticizing OP's standards of beauty. It's probably possible to get actual good art out of midjourney in its current form, but OP's examples are crap and were selected based purely on what was most popular among users.
guajalote
11mo ago
While I agree mostly ... are the top 20% of that page really any worse than 99.9% of popular existing art?
No, it's not any worse, it's about the same, and that's my point. The midjourney stuff is just as crappy as most of the art that gets made today. The "beauty" that OP thinks he's identified is just hyperpalatablity. Unlike most modern art, the midjorney art is inoffensive, but that doesn't make it good or beautiful.
guajalote
11mo ago
The people I was responding to were just as complicit in using metaphors carelessly for impact. It's not exclusively the left's fault that language is a mess. Every culture war meme on the right, "groomer," "murder" (in the case of abortion) etc, are chosen dynamically for impact in the culture war.
In the case of "murder" I think conservatives are using the ordinary definition of the word ("the crime of unlawfully and unjustifiably killing a person") and the disagreement is not over definitions but rather an object level debate about whether abortion is, in fact, murder.
In the case of "groomer" this is clearly a redefinition of the term, and I believe Rufo has admitted this. He popularized the term with the explicit understanding that the term is being redefined in an attempted mimicry or parody of the left's propensity to redefine words.
But I can't think of an example of the right doing what the left frequently does, which is to redefine a term while denying that the term has been redefined. I'm curious if you can think of any examples.
guajalote
11mo ago
I think "socialism" is a good example, thanks. I could quibble with the other examples but I think this one hits the nail on the head.
guajalote
11mo ago
It does sound vaguely plausible, though my priors for "I cured myself with this one weird trick" claims are pretty low so I wouldn't put too much stock in it.
guajalote
11mo ago
Yes, but you can also get a massive improvement via the placebo effect, or just via coincidence, and there will be a huge reporting bias where "one weird tricks" that seem to work will be overreported. So while the phenomenon you describe is also possible, I think the most common situation is simply a mistake on the part of the person reporting their experiences.
guajalote
11mo ago
You focus on the gender dynamics, but I think phenomenon is not primarily about gender.
The two main drivers are, IMO:
-
When two people like each other and spend significant time together, they build trust. This means they will tend to be more charitable toward one another and less dismissive of one another's beliefs. This reduces confirmation bias by making it harder to dismiss the other partner's politics on flimsy grounds, e.g. "they only believe that because they're ignorant/evil."
-
When a person with orthodox politics spends significant time with a person with heterodox politics, on average the person with orthodox politics is more likely to change their views than the person with heterodox politics. This is because it is easy to hold orthodox politics without considering carefully (or even encountering) arguments against those politics, whereas it is impossible to hold heterodox politics without constantly encountering counterarguments. Thus, heterodox beliefs will tend to come into the relationship more "battle hardened" and "stress tested" than orthodox beliefs.
Where gender might play into the dynamic above:
-
Women tend to be more agreeable and thus more likely to hold orthodox beliefs.
-
Women tend to be more agreeable and thus less likely to have engaged in "stress testing" (e.g. vigorous debate) of their political beliefs.
guajalote
11mo ago
I had the extremely good luck of being born as a middle-class American and therefore enjoy a level of privilege that most people at most places and times could only dream of. I grew up with all my necessities taken care of, I got a higher education and postgraduate degree, I had access to all the fruits of modern technology - antibiotics, air conditioning, the internet. I have daily use of things that many kings of old would have traded half their kingdoms for. That I would have all the privileges I enjoy is exceedingly unlikely, I am among a tiny fraction of a percent of the most privileged human beings who have ever lived on earth.
Not only that, but most members of this very forum are similarly privileged. The majority of users here are middle class or higher, educated, and live in conditions that most human beings could have never even dreamed of. What are the odds that hundreds of people, all from among a tiny fraction of a percent of the most privileged humans in history, would all find themselves here at some random obscure internet forum? We are talking about a tiny fraction of a percent, multiplied by a tiny fraction of a percent, multiplied by a tiny fraction of a percent, repeated hundreds of times. We're talking about odds of some miniscule fraction like 0.0000....0001%.
Therefore, I submit that The Motte was created by Jesus Christ himself. The odds that a place like this could arise by the chance congregating of individuals is so astronomically unlikely that we can dismiss such a hypothesis as ludicrous. Only the guiding hand of our Lord and Savior could have created such a rare and perfectly fine-tuned set of conditions.
guajalote
11mo ago
Your point only functions as an explanation of fine tuning if we assume in advance that the (or “a”) multiverse hypothesis is true.
No, it doesn't. It works equally well with a one-shot universe. If the parameters had not been properly fined tuned, then we wouldn't be here. Therefore we can only observe a universe in which the parameters are fine tuned. This is true regardless of how many universes exist.
It’s unlikely that I would result from my parent’s act of conception, but billions of acts of conception were happening before I was conceived.
Those other billions of acts of conception are irrelevant. You could not possibly have been made by any of those other acts of conception, since those other people had different genes than your parents. Moreover, the statistical likelihood of you being conceived is independent of each prior act of conception in the same way that the outcome of a coin flip is independent of prior flips.
The relevant analogy is to note the large number of eggs and sperm your parents produced over their lifetimes and the extreme unlikelihood that the particular egg and particular sperm that produced you would have combined. This was, indeed, extremely unlikely ex ante, but you can only observe an ex post world in which this highly unlikely event did in fact occur.
guajalote
11mo ago
In a world where each subatomic particle is stamped with the words "made by Jesus Christ," we have two apparently unlikely things being true:
-
The parameters of the universe are fine-tuned to produce human life, and
-
The parameters of the universe are fine-tuned to reference a particular Hebrew carpenter by name.
Even though point 1 seems extremely unlikely ex ante, it is not actually unlikely given that human observers exist. Point 1 must necessarily be true in any universe in which human observers exist. Therefore, observing that point 1 is true does not tell us any information about how our universe came into existence, because point 1 is 100% guaranteed to be true in any universe we can observe.
Point 2, on the other hand, is actually extremely unlikely. There is no reason why point 2 must be true in every universe where human observers exist, and in fact we know it need not be true in all such universes because it isn't true in our own universe. Therefore point 2, if it were true, would actually be powerful evidence of something (perhaps the truth of Christianity, or perhaps that we're living in a simulation and are being messed with by the simulators).
guajalote
11mo ago
For example, sometimes people seem to be suggesting that because a phenomenon involves the creation of observers, the phenomenon requires no explanation, which is silly.
I am not saying that a phenomenon that involves the creation of observers requires no explanation, I am saying that observing such a phenomenon exists provides no information because such a phenomenon is 100% guaranteed to exist in any universe with observers. So I am saying we cannot draw any useful conclusions from observing such a phenomenon exists.
For example, every person I meet who has ever been skydiving tells me that their parachute has opened every time they've gone skydiving. This fact tells me no information about how likely parachutes are to open (except that the probability is not 0%). It provides no information about parachutes one way or the other.
guajalote
11mo ago
I think your understanding is incorrect. Evolving multicellular organisms once you already have unicellular organisms is probably much easier than abiogenesis; it has in fact happened multiple times independently. Multicellularity has evolved independently at least 25 times in eukaryotes, and also in some prokaryotes, like cyanobacteria, myxobacteria, actinomycetes, Magnetoglobus multicellularis or Methanosarcina.
How does a cell that evolved to be all about itself and it's direct descendants ever decide to team up with several other cells, which all abandon their individuality and dedicate themselves to the survival of a higher-order organism?
The same reason multicellular organisms evolve social behaviors - your relatives share a high proportion of your genes, so it is adaptive for genes to code for traits that improve your relatives' survival and propagation.
guajalote
10mo ago
It doesn't provide information, because the only possible one-shot universe we can observe is one with observers. Maybe there was a 0.000...01% chance that the one-shot universe would be fine-tuned for observers, or maybe there was a 99.999...% chance that the one-shot universe would be fine-tuned for observers. Either of these possibilities is consistent with the fact that we, as observers, see a one-shot universe with observers in it.
You, the observer, notice, hey, I'm a rational being who came into existence! I'm saying that it's rational to think that this should update your priors towards hypothesis 1 over hypothesis 2. But only if there's a one-shot or few-shot universe.
No, it shouldn't change your priors, irrespective of whether there's a one-shot or multi-shot universe.
Given that you are a rational being, the odds that you will observe a universe where a rational being came into existence are exactly 100%. This is true regardless of whether hypothesis 1 or hypothesis 2 is true, and therefore it tells you no additional information about which hypothesis is correct.
You wake up. Assuming there's not going to be any distinguishing sensation between the two ways you could wake up, which should you think is more likely? I would think you should think that there's a 50:1 chance that it worked.
That's true, because this situation is materially different from the one we are talking about above. Here, we know two sets of probabilities ex ante (both of which can occur), and are now trying to decide, ex post, which is more likely to have occurred. Given two different possibilites, the one with the higher probability is, by definition, the one that was more likely to occur (this is true whether the surgery is one-shot or many-shot, by the way).
In the situation we are discussing, we don't know anything about the probabilities ex ante, and we are trying to derive those probabilities based on our ex post observations.
A better analogy would be: you go into a surgery and nobody knows your odds of survival. You wake up after the surgery. What, if anything, does this tell you about your ex ante odds of surviving the surgery? The answer is, it tells you nothing about those odds. It just tells you that you survived. Your odds of survival could have been 0.001% or 99.9%, but since you can only observe outcomes in which you survive, that fact that you observe your own survival gives you no additional information about the ex ante likelihood of that outcome.
Another example to illustrate the point. Suppose an alien hands you a black box with a screen and a button on it. You push the button, and the number "21" appears on the screen. Pushing the button again does nothing and you cannot disassemble the box to learn how it works. What are the odds that the box was going to display the number "21" when you pushed the button? The answer is, you have no idea (except you know the odds are not 0%). It might have been a 100% chance, it might have been a 0.000000001% chance. You have no way of knowing based on your single ex ante observation.
guajalote
10mo ago
Starting from an objective not-taking-into-account-yet-that-you-exist estimate of likelihood that a random universe would be able to contain life, you should have some estimate of a likelihood that a universe could contain life.
If we could start there then we would gain some information from the later observation that the universe contains life. But we cannot start there. We start in a universe where the existence of life is a given, with 100% probability.
But now that there's some agent, that's data! That's information!
It's not. You can only observe a universe with agents because you are an agent. That statement is always true, no matter the prior probabilities, so you cannot draw any conclusion about prior probabilities from the fact that it happened.
what I just said requires knowing how often other people survive surgery
Right, if we could bring in outside information about how likely a universe is to contain life and what factors influence that, then we might be able to draw some conclusions from the fact of our existence, but we have no such outside information, so we cannot draw such conclusions.
guajalote
10mo ago
Why can't we start there? Isn't that equivalent to stating that you can't think about universes where life doesn't exist? That's transparently false. Working out what our beliefs should be if we ignore a piece of information is something we're allowed to do.
I'm saying we literally can't start there. We can't go back and observe before the beginning of the universe. We don't have any information about what was happening then, or what the dynamics of the situation looked like. We only have information about the universe we currently inhabit.
We can certainly think about it, but we don't know anything about it, so it doesn't do us much good.
No, that statement isn't always true. It's only always true for observers. That means that you should shift your probability mass from what they would be if you ignored which worlds you're more likely to exist in as an observer, to what they should be after taking that into account.
You are an observer, so it's always subjectively true for you 100% of the time.
Assuming our thinking is at all Bayesian, shouldn't we have some sort of probability distribution? Not sure exactly what one should look like, but that should exist.
Yes, so start with whatever your Bayesian priors are. You think there's an x% chance that the universe would contain observers. For every possible value of x other than x=0, the odds that you will observe a universe with observers is equal to 100%. So there is no basis to either raise or lower your prior based on this observation, because the odds of it being true are exactly the same in every possible scenario other than x=0.
In any case, did you miss what I said about how it should always be the case, whatever that probability distribution is, that you should update (ignoring other post-waking-up information, on your existence alone) towards the chance of your survival having been higher than you thought it was before your surgery?
Assuming you have no outside information about the surgery, there is no basis to update your priors. This is true for the same reason I explained above. For any prior other than x=0, the chance that you will observe you survived the surgery is exactly equal to 100%. If you did not survive, you would not make any observation, so the only possible observation you can make is "I survived."
If you're saying that we always have to take into account that we exist, we can't think about cases where that's not true, how does it not follow that you can't imagine worlds where you don't exist?
You can imagine worlds where you don't exist, but imagining such worlds doesn't tell you anything about how likely they are.
And worlds with observers will look more like multiverses, there is some reason why the universe's fine-tuning is necessary, or theism, because in our ideas of the possibilities, observers are relatively more likely to occur in worlds of those varieties, as compared to one-shot worlds that require high degrees of fine-tuning.
This is the claim I am pushing back against. We have, broadly, three categories of possibilities:
-
God created universe(s)
-
Natural processes created multiverses
-
Natural processes created a one-shot universe
We have no information about which of these scenarios are even possible to begin with. Even if each is possible, we don't know which one is more likely to create observers. So let's naively assign a 33% probability to each. Now we are going to make empirical observations and see if that moves our priors.
Given that you (an observer) exist, what is the chance that you will empirically observe a universe with observers in scenario 1? Answer: 100%.
Given that you (an observer) exist, what is the chance that you will empirically observe a universe with observers in scenario 2? Answer: 100%.
Given that you (an observer) exist, what is the chance that you will empirically observe a universe with observers in scenario 3? Answer: 100%.
So after making the observation, our priors should remain the same, 33% for each scenario, because our observation is equally likely to occur in each scenario.
No, it isn't 100%. You're ignoring all the scenarios where you don't exist. Yes, you won't be around in those worlds to decrease your estimate of the probability of life existing, but that doesn't mean you shouldn't take them into account.
You're ignoring what I said: "the odds that you will observe a universe with observers is equal to 100%." There may be some probability of a universe without observers existing, but you can never actually observe this, and you therefore have no information about how likely it is to occur. You have no way to "take this into account" because you don't know if this scenario can even exist, or how likely it is.
And we've estalished that we're talking about a one-shot universe, so there's definitely a good chance that there is no observer, in which case the odds taht you will observe a universe is less than 100%, because in some of those worlds you don't observe anything at all.
First of all, it's not a given that the odds of observers are less than 100%. They are less than or equal to 100%. It's possible that a one-shot universe must necessarily create observers. We have no clue about the underlying physical process that would create such a universe and therefore we have no clue about the likelihoods. There may be no randomness involved.
Second, the odds that you will observe a universe with observers is 100% because the word "you" in that sentence necessarily implies the existence of at least one observer. The statement "in some of those worlds you don't observe anything at all" is incoherent, because there is no "you" in such worlds. The statement "you will always observe observers" is a tautology, it is necessarily true based on the definition of the word "you."
To check that that makes sense, we indeed find that 89% of rational agents who thought this way and woke up had surgery that worked.
My point is that here, in the universe scenario, we have no way of testing whether the agents are correct or wrong. In the surgery scenario we can look back and ask the doctor "did the surgery work" and find out whether the agents' guesses were right or wrong. Here, there is no way to check the truth value of the agents' guesses, and therefore no way to find out if their priors are correct or way off.
A better analogy would be: suppose you go in for surgery, and no one has any clue about the likelihood of success or the likelihood you will wake up. So let's say the odds of success are x%, the odds of failure are (100-x)%, the odds you will wake up if the surgery's successful are y%, and the odds you will wake up if the surgery's a failure are z%. You're a Bayesian so you assign arbitrary probabilities to x, y, and z, say 50% for each. You go in for the surgery and you wake up after. You are provided no additional information. How, if at all, should you adjust your priors?
And since inhospitable planets are so much more common overall, I would think, shouldn't you be shocked that we're on a hospitable one? You clearly don't actually think that, but I don't see how that's different from the case that we're arguing over.
The difference is, we know that hospitable planets are relatively rare. We have external knowledge about how planets form and can observe both planets with and without life. We didn't come to this conclusion based on the fact of our existence. We came to this conclusion by observing the external universe.
Imagine we had zero information about the universe beyond planet earth. Should we assume that planets with life are rare based on the fact that humans exist? Should we assume that earth is especially hospitable for life? Should we assume other planets even exist at all? In the absence of any external information, it's totally possible that planets with life are common and that earth is uniquely hostile to life compared with those other planets. Or it's possible that earth is the only planet. We would have no way of knowing, based merely on the observation that we exist.
All these are to the one-shot, heavy-fine-tuning-required-that-we'll-just-have-to-luck-into world what Earth is to Pluto.
No, because we have no idea how "unlikely" the fine-tuning of our universe is. We have no reason to think that dice were rolled. It's possible that what seems to us like fine-tuning is actually just some necessary constraint of an underlying physical system that created the universe, such that the odds of fine-tuning were 100%. Or its possible that a fine-tuned universe was super unlikely and we just got lucky. We have no way of knowing.
Likewise, there's no reason to assume that God likes creating fine-tuned universes. He might love rolling dice and creating trillions and trillions of dead universes for every living one. After all, he seems to love creating uninhabited planets, so why not uninhabited universes?
guajalote
10mo ago
If you disagree with my assertion, it’s actually super-duper easy to refute it; all anyone needs to do is offer up a coherent description of either cis or trans gender identity void of any reference to gender stereotypes. ... A single point cannot resonate or clash with itself, as these dynamics necessitate interaction between distinct elements.
Such a description exists and used to be in vogue, though it seemingly no longer is.
The two points of comparison are both allegedly biological: (1) reproductive sex (i.e what gametes one produces), and (2) mental sex. The claim is that there exists a sexually bimodal distribution of mental traits that is mediated primarily by biology (i.e. it's not primarily driven by gender stereotypes) and that usually correlates strongly with reproductive sex. A trans person is someone whose mental traits break from this typical correlation, i.e., their mental traits fall in the opposite mode of the bimodal sexual distribution.
I don't know whether this framework is accurate, but it's at least plausible and coherent.
guajalote
10mo ago
That may be true, but I'm not trying to argue the object level point. I'm just saying that there exist definitions "trans" that wouldn't run afoul of ymeskhout's objections.
guajalote
10mo ago
If mental sex exists as a biological reality, then it would refer to a fact about normal biological development independent of culture or stereotypes. It would be no more of a stereotype than statements like "men are stronger than women" or "women have larger breasts than men." The fact that these statements are not true 100% of the time doesn't make them stereotypes.
Couldn't agree more. In addition to Yud's failure to communicate concisely and clearly, I feel like his specific arguments are poorly chosen. There are more convincing responses that can be given to common questions and objections.
Question: Why can't we just switch off the AI?
Yud's answer: It will come up with some sophisticated way to prevent this, like using zero-day exploits nobody knows about.
My answer: All we needed to do to stop Hitler was shoot him in the head. Easy as flipping a switch, basically. But tens of millions died in the process. All you really need to be dangerous and hard to kill is the ability to communicate and persuade, and a superhuman AI will be much better at this than Hitler.
Question: How will an AI kill all of humanity?
Yud's answer: Sophisticated nanobots.
My answer: Humans already pretty much have the technology to kill all humans, between nuclear and biological weapons. Even if we can perfectly align superhuman AIs, they will end up working for governments and militaries and enhancing those killing capacities even further. Killing all humans is pretty close to being a solved problem, and all that's missing is a malignant AI (or a malignant human controlling an aligned AI) to pull the trigger. Edit: Also it's probably not necessary to kill all humans, just kill most of us and collapse society to the point that the survivors don't pose a meaningful threat to the AI's goals.
More options
Context Copy link