astrolabia
6hr ago
Can you help me understand this claim more concretely? E.g. if an LLM had just successfully designed a bridge for me, but then I modified the design to make it not useful in some way, for some kinds of changes it wouldn't be able to tell if my change was good or not? But a human would?
astrolabia
2d ago
I agree that alignment is easy in the sense of getting models to understand what we want, but it's far from clear that it's easy in the sense of making models want the same thing. RL models reward hack all the time.
What on earth makes you think instrumental convergence "doesn't actually happen"? It happens all the time, e.g. by reward hacking or sycophancy! It's almost the definition of agency!
Neuralese is a myth? What is that supposed to mean? RL on soft tokens is an active area of research and will almost certainly always work better (in the sense of getting higher rewards) than using hard tokens everywhere.
astrolabia
3d ago
none of it is going to happen in the way the AI safety movement predicts
Care to elaborate? What kinds of things do you think are going to happen differently than the AI safety people think?
astrolabia
26d ago
Is this a bit? Yes collecting a dataset is tons of work, but tokenizing it is trivial.
astrolabia
28d ago
I agree with everything you wrote in this reply. But your reply seems to have nothing to do with your message I originally replied to. Why were you mentioning the cost of tokenization?
astrolabia
29d ago
Why are you talking about the footprint of a tokenizer? Tokenization is cheap compared to actually evaluating the LLM.
astrolabia
1mo ago
Thanks for clarifying your position. I suppose if I thought that ChatGPT was a one-off, I might have a similar position to yours. In my view, however, there's been a pretty consistent, smooth, and somewhat predictable trajectory that whole time, and has continued since ChatGPT. If every almost single remaining eval being almost saturated from ChatGPT till now doesn't look like "anything to write home about", I don't know what could.
astrolabia
1mo ago
I'm hoping you'd providing arguments or evidence about the likelihood of different outcomes. I'm not sure what calling something a null hypothesis means other than being a bald assertion that it's likely.
I'll go first: rapid recent improvements in AI continuously over the last 12 years and massive R&D efforts going on make it likely that substantial improvements will continue.
astrolabia
1mo ago
Seems like you're just begging the question here. Why is that the correct null hypothesis?
astrolabia
1mo ago
There are just as many people committed to cracking eggs at all costs as there people who will claim that puberty blockers gave their cancer cancer.
I don't think that's true. Or at least, my impression is that almost every elementary through high school teacher in north america who talks about the issue gives the impression that it's basically possible to successfully transition.
All I can say is that we should let people make their own choices, and if they're hard and risky choices, do our best to ensure they're exposed to the facts they need.
I don't think I'm willing to bite the libertarian bullet here. E.g. I don't want my kids to have the option to do heroin, even if it's paired with a pamphlet explaining the real likely outcomes. However, I don't even think that that's a viable option. Seems like our options are: ban and demonize heroin, or legalize it and subsidize its use (as was recently done in British Columbia).
Same with transitioning kids: I don't see how we ever get to a world where it's both legal and the pros and cons are presented honestly. So I think I'd rather throw the few kids who could conceivably benefit from it under the bus and ban it for everybody.
astrolabia
1mo ago
Sounds like we agree on basically everything. Except I want to reserve the right to value things independently of whether they cause secondary problems. E.g. I'd fight to stop secretly torturing people even if the practice didn't cause secondary problems.
astrolabia
1mo ago
I basically agree with you about values and freedom. I guess my main fear is around the information environment we provide re: "This is the closest we can get you today." I'm not an expert but I get the impression that many (maybe most?) people who attempt to transition are deeply mislead about both the best and worst-case outcomes. I just don't expect any modern Western institution to be able to honest about what wretched results most transitioners end up having, nor about what most people honestly think of them.
Relatedly, Blanchard wrote about how his MtF patients could usually see that the other MtF patients clearly did not pass, but believed that they themselves did.
I think you could make similar arguments about the information environment surrounding lots of other early life choices, or educational choices such as pursuing arts degrees. But most of those are less catastrophic and irreversible. I guess at least Western society now does a pretty good job of showing the downsides of joining the army.
astrolabia
1mo ago
how to accurately determine if a child is trans
It seems like this would require defining what it means to be trans... any suggestions?
Those people ought to just be ignored and will simply lose their credibility over time if giving puberty blockers to kids proves itself to be fine.
Seems like you're begging the question here. If people think something is wrong in principle, then it won't be "fine" by their lights even if it doesn't cause secondary problems.
astrolabia
2mo ago
My guess is that part of the idea is to route around management. Presumably do-nothing employees are already known to their managers, but have been receiving some sort of protection for years.
astrolabia
2mo ago
One reason to be part of a pension like the NHS is that it puts you in an alliance with a large constituency who might plausibly have enough political power between them to keep the gravy train going down the road.
astrolabia
2mo ago
Neither wants to consider that getting off his ass and doing things will solve his problems, or at least make them manageable.
Seems like doing both that and addressing whatever seems to be a larger problem is possible, and in fact, laudible? Yes many people ignore their own problems too much while ineffectually preening about global problems, but I guess I'd also hate the world where no one had the impulse for public service. OTOH, now that I think about it, that might look like a libertarian paradise if there were still kickstarter-like coordination mechanisms.
astrolabia
3mo ago
Got it. But what's the new, non-outdated consensus?
astrolabia
3mo ago
Right. But what's the new consensus? That AIs will take all our jobs, including the police and military, and we'll all live happily off UBI while contributing nothing, and no one's ever going to take our stuff away?
astrolabia
3mo ago
The AI doomers are only an extreme example of how completely antiquated the old view is.
Can you elaborate? What do you think the doomer position is?
astrolabia
4mo ago
I agree that setting the precedent of meddling with family formation is a bad one. I'm just saying that I don't understand what your advice looks like in practice. If my local municipality proposes subsidizing building a daycare, how do you vote?
astrolabia
4mo ago
I agree that individual returns to societal-level advocacy are usually small, but again I don't understand where you draw then line between "advocacy for strong families" versus "Attempting to optimize policies for the societal production of kids".
Having more kids always results in having more kids. Raises are to get market value for my labor, not because I have kids.
If having kids is so central, then why spend time trying to get market value for your labor, instead of spending that time having more kids?
nonintervention might result in ethnic replacement or demographic collapse, but these are common enough over recorded history that I don't have any personal problem with it.
Something bad being common doesn't make it OK - it makes it scarier! And both of these things increase the chance that your descendants won't be able to have as many kids as they otherwise would.
astrolabia
4mo ago
I don't understand the distinction between working on having your own kids versus advocating for policies that'd make it easier for you and yours to have more kids. Surely you'd advocate for a raise to help pay for your own kids? How about for lower taxes at a municipal level? How about per-kid payments at a federal level?
astrolabia
5mo ago
Man, lately Tyler just seems so off the mark. He keeps talking about AGI in the most head-in-the-sand, dunky and dismissive way without even making clear claims or any concrete arguments. I think AGI doom is one of the most mind-killing topics out there amongst otherwise high decouplers.
astrolabia
5mo ago
Fair enough, but I give him partial credit for asking for a withdrawl, though I don't know any details.
It really is remarkable the strength of claims that otherwise smart people will make about the impossibility of AI doing something. As evidenced by IGI's reply, I think usually if someone has gotten this far without updating, you shouldn't expect a mere compilation of strong evidence to change their minds, but just to prompt the smallest possible retreat.
I had an amazing conversation with an academic economist that went along similar lines. I asked why his profession generally wasn't willing to even entertain the idea that AI could act as a substitute for human labor, and he said "well it's not happening yet, and making predictions is beyond the scope of our profession". Just mind-boggling.
To empathize a little, I think that people intuitively understand that admitting that a machine will be able to do everything important better than them permanently weakens their bargaining position. As someone who hopes humanity will successfully form a cartel before it's too late, I fear we're in a double-bind where acknowledging the problem we face makes it worse.
More options
Context Copy link