MathWizard
Good things are good
No bio...
User ID: 164
I'm not sure how you could be confident of that because the entire point of "fast takeoff" is the nonlinearity. It's not saying "AI is going to improve steadily at a fast pace and we're going to get 10% of the way closer to lightcone speed each year for 10 years. It's "At some point, AI will be slightly smarter than humans at coding. Once it does that, it can write its own code and make itself smarter. Once it can do that, growth suddenly become exponential because the smarter AI can make itself smarter faster. And then who knows what happens after that?"
I'm not 100% convinced that this is how things play out, but "AI is gradually but slowly getting better at coding" is weak evidence towards that possibility, not against it.
I'm not going maximally extreme and saying it "nullifies the agent's right to self defense". But I'm pointing out that they seem to be deliberately exploiting the right to self defense by putting themselves in danger in order to be allowed to defend themselves. There's circular shenanigans going on here where they make themselves less safe, going against the spirit of the law (which is intended to protect them) in order to trigger the letter of the law and get what they want (the right to shoot the criminal if they try to flee, which the law ordinarily does not give). The agent violates their own rights in part in order to then recover them in a manner with useful side benefits. I'm not saying the law should say "if an agent stands in front of a car oops I guess they have to let themselves die now". But clearly something has gone wrong if the law intended to make them more safe is encouraging them to make themselves less safe.
There are a number of differences. First, the car is both the weapon and the means of transportation. The chef could easily drop the knife and then charge the police officer which, while they definitely should not do, would not be deadly force and not deserve death, even if it does deserve harsh punishment.
Second, the police officer has a legitimate means of stopping the chef by physically blocking the door. Because people can stop people, but people cannot stop vehicles. The police officer fully expects that if the chef comes at him he can physically restrain him. The police in front of a car does not intend this. The officer does not have any means of preventing escape other than their gun. Their body is not going to stop the car, they don't expect their body to stop the car. They do not intend to physically restrain the car, and they very dearly hope they don't have to try. If they did not have a gun or were not allowed to use it they wouldn't stand there in the first place because they're not stupid and they don't want to die. The only reason to stand in front of a car is to threaten the suspect with a gun. It is not a restraint it is a threat.
The difference is that an officer physically grappling them physically restrains them. The officer has a plausible means of preventing the escape beyond their gun. If the officer did not have a gun, or was not allowed to use their gun, a physical grapple is still a useful and legitimate means of restraining a suspect. A normal, non-police officer attempting to do a citizen's arrest might plausibly physically restrain someone this way because it literally restrains them.
In the car case, the officer does not have any means of preventing escape other than their gun. Their body is not going to stop the car, they don't expect their body to stop the car. They do not intend to physically restrain the car, and they very dearly hope they don't have to try. If they did not have a gun or were not allowed to use it they wouldn't stand there in the first place because they're not stupid and they don't want to die. The only reason to stand in front of a car is to threaten the suspect with a gun. It is not a restraint it is a threat.
I'm not saying people actually have a right to flee. They're still breaking the law. I'm saying their fleeing is not equivalent to violence and deliberately booby trapping their flight path to be deadly is wrong. Ie, imagine the police officers were going to bust into a drug house but, before entering, they stick landmines at all of the doors and windows so anyone fleeing gets blown up. Yeah, the drug dealers should get arrested and don't deserve to escape. But if they try to flee they shouldn't die for it. I'm pro-death penalty for especially horrific acts of villainy. I'm pro police officers killing people if forced into a dilemma where it's their life vs the life of a criminal threatening them. I'm not pro killing literally any criminal for literally any crime. Consequences should be proportional. Fleeing is not proportional to death. Police officers endangering themselves in order to create an artificial escalation so that fleeing is proportional to death is not the fleeing criminal's fault, but the police's, so does not change the moral calculus here.
You say this as if this is not already the case in our current reality. How exactly do you think that police use of force laws work? Because I guarantee you it's not the free for all that anti-police activists like to think it is.
The "almost" equivalent is the part where the neck garotte would probably be illegal in our world, but is legal in this hypothetical.
Nobody has a legal or moral right to flee from the police, nonviolently or otherwise! Preventing criminals from fleeing the police is a good thing! They shouldn't do that! Why do you seemingly care so much about making sure that criminals have a fair shot at beating an arrest?
I'm not saying people actually have a right to flee. They're still breaking the law. I'm saying their fleeing is not equivalent to violence and deliberately booby trapping their flight path to be deadly is wrong. Ie, imagine the police officers were going to bust into a drug house but, before entering, they stick landmines at all of the doors and windows so anyone fleeing gets blown up. Yeah, the drug dealers should get arrested and don't deserve to escape. But if they try to flee they shouldn't die for it. I'm pro-death penalty for especially horrific acts of villainy. I'm pro police officers killing people if forced into a dilemma where it's their life vs the life of a criminal threatening them. I'm not pro killing literally any criminal for literally any crime. Consequences should be proportional. Fleeing is not proportional to death. Police officers endangering themselves in order to create an artificial escalation so that fleeing is proportional to death is not the fleeing criminal's fault, but the police's, so does not change the moral calculus here.
It's the far extreme on a spectrum of "deliberately put oneself in harms way that the suspect did not themselves intend to put you under". If you barge into a restaurant kitchen and the chef is holding a knife and you dive underneath him, he is not threatening you with the knife. You threatened yourself. Millions of people drive cars. Technically they are deadly weapons but they aren't generally going around threatening people with them. If you jump in front of a moving car then the driver is not threatening you, you are threatening yourself with it.
If you jump in front of an unmoving car then there's some ambiguity there. But if your goal of moving in front of it is with the purpose of threatening yourself with it (the police don't expect their body to stop the car, they expect their guns to stop the car) then something fishy is going on. From the misbehaving police officers perspective, the car's status as a weapon is a feature, and the policeman's vulnerability is being leveraged this way. If the police had magical invincibility powers that made them unharmed by getting hit by cars the strategy would no longer work. We want to incentivize police officers to keep themselves more safe, not incentivize them to endanger themselves to exploit laws intended to protect them. Clearly something has gone wrong when that has become the case.
I'm not entirely sure where I stand on this issue, but to push back on the idea of it being a slippery slope, I think we can steelman the "fleeing the police shouldn’t be a death sentence" idea to something like "the police should not deliberately block off only nonviolent methods of fleeing in order to force an equivalence between fleeing and violence.
Imagine a dystopia in which police have a secret goal of wanting to shoot as many people as possible, but are legally prohibited from this because their laws are almost equivalent to ours: you can only shoot someone in self defense (or defense of another), but have some extra loopholes that allow the following scenario. The police always travel in pairs, and instead of normal handcuffs they carry one cuff with a long thin wire dangling off them. When a police officer cuffs someone it doesn't directly restrain them in any way, but the police officer ties the wire around their own neck. This means if the suspect attempts to run and gets far enough away, the wire tightens and slices/strangles the officer. The other officer can then legally shoot the suspect in order to save their partner's life. That is, the officer is deliberately endangering themselves in a conditional way in order to create opportunities to shoot people.
The steelmanned argument would then place "standing in front of a driven vehicle" in this same scenario. You are not physically restraining a person. You are not actually preventing them from escaping. Instead, you are creating a scenario in which you deliberately endanger yourself conditional on them fleeing as an excuse to shoot them. This is roughly equivalent to just training a gun on them and saying "don't run or I'll shoot you", which police officers are generally not allowed to do. This is a loophole in which they are allowed to do it. Saying "we should close this loophole, you can't just put yourself in danger for the express purpose of giving yourselves opportunities to shoot people" does not slip into "violence is allowed" because it's categorically and consistently anti danger/violence. It's not necessarily about deliberately giving people opportunities to flee, or even failing to close off opportunities to flee if you can actually do that, but it's a claim that abusing your legal power and using yourself as a hostage is not a legitimate means to close off escape.
Of course, I expect a large fraction of people do believe weaker versions of this and just hate police. But I think there is some legitimate point here in the stronger version.
The health of the market relies on the wisdom of crowds, which requires crowds of people to be able to reliably win from it. Insider trading is bad not because in some moral sense "unfairness" is bad, but because if it happens often enough that ordinary people learn that it's unfair they'll stop participating. Prediction markets are zero sum to begin with, so I don't expect them to survive long-term without subsidies, but what life they do have is built by the belief that smart people can earn money from their intuitions. If that fails to be true because insiders keep swooping in and snatching up all the money at the last minute, then fewer non-insiders will participate, and we'll only ever get accurate results when there are insiders.
This would be less catastrophic than if it happened to the stock market, since the death of prediction markets wouldn't ruin us the same way the death of capital investing, but it's still a potentially existential crisis within its demand. This isn't just about people's moral intuitions, there are stakes.
- Prev
- Next

What? We have basically no forms of self-modification available whatsoever. You can study and reason, I guess, which is vaguely like adding training data to an AI. You can try Eugenics, but that's highly controversial, incredibly slow, and has not been tried at scale for long enough. Hitler tried and then people stopped him before he could get very far. Gene editing technology is very new and barely used due to controversy and not being good enough and taking decades to get any sort of feedback on.
We have NOT been "trying to make ourselves smarter" in the same way or any way comparable to an AI writing code for a new AI with the express purpose of making it smarter. What we have been doing is trying to make AI smarter with more powerful computers and better algorithms and training and it has worked. The AI of this year is way smarter than the AI of last year, because coders got better at what they're doing and made progress that made it smarter. If you have more and better coders you get smarter AI. We can't do that to humans... yet. Maybe some day we will. But we don't have the technology to genetically engineer smarter humans in a similar way, so I don't know what sort of comparison you're trying to make here.
More options
Context Copy link