SnapDragon

SnapDragon 21hr ago

Since I generally respect you and your posts, I want to try this one more time. I don't necessarily buy that we should just declare this a "fundamental values difference" and say that we're now beyond any hope of rational agreement. And while you may have "coherent reasons for your position", that can be true of many evil ideologies. Evil =/= incoherent.

You brought up preferences, and I get the impression that you pattern-matched my ideology to a Rawlsian one that you should never prefer your own tribe, which is an extreme that I definitely don't hold to. I prefer my own happiness over others to a decent extent, and that goes for my family, my friends, my nation, and my species. I'm not asking you to give up that preference! Self-interest is the glue that holds a society of individuals together, and capitalism's magic is that it doesn't try to deny it, merely harnesses it in a way that doesn't degenerate into misery for all. I just don't think that preference should be infinitely strong: Beings in your outgroup should still matter more than zero. You shouldn't torture them horribly for a tiny gain, even if there are no repercussions. You should prefer a world where you're happy and they're happy.

You didn't respond to my main concern, which was that yours is the same "coherent" reasoning that led to many racial atrocities in the past. It doesn't seem very universally defensible, and often leads to horrific outcomes, when you simply draw a circle around whoever you know growing up and declare that this is the circle of beings that hold moral worth. Do you think Hitler's only mistake was that he drew the circle around "Aryans" instead of around humans? Or the African slavers who drew it around "Europeans"?

We currently live in a society where there's no friction between your ideology and mine, because humans are the only sapients around. (I'll set animal suffering aside, because I'm ambivalent on it too.) But it's very possible that, within our lifetimes, it will suddenly matter deeply, where our society will consist of both humans AND sapient AIs. All I'm asking is that you give some moral valence to the suffering of beings that are outside the circle you've drawn. Not zero, not infinite. It's a low-cost alteration to your ideology, and it stops there, I'm not trying to, uh, whatever the opposite of murder-Gandhi is. And if some of our ancestors had made the same small concession, so much misery could have been avoided.

Context

SnapDragon 2d ago

I'm an unabashed transhumanist chauvinist, I think that only humans and our direct transhuman and posthuman descendants or derivatives deserve rights. LLMs don't count, nor would sentient aliens that we could beat by force.

Huh, I'm pretty surprised to hear this, and I have a deep ethical disagreement with you here. In my opinion, what is special and valuable about humans - and the thing that fundamentally gives value to the universe itself - is sapience. But we should cherish it just as much in a different form. (I mean, I agree LLMs don't count, but that's just because I see no way they, lacking persistence of thought, could actually be conscious.) Where does this bright line surrounding us "humans and descendants" come from? In a different era, your argument would easily pattern-match to arguments about subjugating other races instead. Why do black people now have moral valence, but some alien from Alpha Centauri wouldn't?

I'm not an expert in philosophy, but I do think there are solid arguments for acting this way (e.g. the categorical imperative). Just like I'm an atheist who still doesn't act like an immoral sociopath when I can get away with it, I think we as a species should not be focused only on our own well-being at the cost of all other intelligent species. Not because of the threat of punishment, and not even because I hope any aliens we meet would similarly value our well-being in a way that you wouldn't. But because existence will just be a better place if we can all get along and not act as game-theory-optimizing selfish machines, and I'm willing to work towards that.

BTW, I don't think your eating-a-pig example is a good one. It's irrelevant to the pig what we do after killing it. A better question is, would you be fine with torturing a pig while it's alive?

Context

SnapDragon 2d ago

No, but somehow these days they're tuning their final models to get to "I don't know" anyways. Maybe they're not just glorified autocomplete? 10 months ago was the first time I got an LLM to admit it couldn't answer a question of mine (although it did still make helpful suggestions); not only did the other models back then give me wrong answers, IIRC one of them went on to gaslight me about it rather than admit the mistake. (two years ago this gaslighting would have been the rule rather than the exception) IMHO that "I don't know" was the exact point at which AI started to have positive utility for me. Sometimes an AI still isn't helpful, but it's at least often worth throwing a problem at now, not a waste of time.

That's good to hear. I'm not saying that the hallucination problem can't be mitigated, I'm just saying that it's a struggle and it's likely to continue to be a struggle, even if LLMs continue to get smarter for a long time. The way I think of it - which is definitely an oversimplification but possibly a useful one - is that next-token prediction really isn't the kind of intelligence that we wanted to develop, but it's what we discovered first. So in some ways - keeping models focused on tasks, preventing malicious usage, learning in real time, avoiding hallucinations - we're paying the cost of trying to pound that square peg into a round hole. With enough effort, paying enough training/inference costs, we often can do it. But perhaps at some point we'll discover a different framework for AI that better matches our own sapience at lower cost.

Context

SnapDragon 2d ago

Here's a relevant AI Explained video about Mythos. Some highlights and personal comments:

7:28 Right now I think coding models are at their most powerful when being used as a force multiplier for human experts. (Akin to Cyborg Chess.) Here, a computer security expert mentions that he found more vulnerabilities in a few weeks than in his entire prior career. This ability to find zero-day exploits isn't an artificial benchmark, this is a real-world result that shows we really are entering some sort of new regime. Although ... I suspect statements like this are going to get so common that we no longer recognize how startling they are, like how we ignore the fact that models flawlessly understanding natural speech would have been considered miraculous 10 years ago. And we'll get more idiotic posts by so-called "skeptics" who think that spending 30 minutes failing at using AI counts as definitive proof that frontier models do not exhibit intelligence.
9:10 Safety concerns related to some prior discussion with @Corvos, @YoungAchamian, @roystgnr, and others. To quote: "In contrast, experts were consistently able to construct largely feasible catastrophic scenarios, reinforcing a view of the model as a powerful force-multiplier of existing capabilities." We're not close to the point of plagues being bioengineered in garages, fortunately, but at some point a reasonably-sized terrorist group with some funds and some expertise might be able to do a lot of damage.
13:23 I really don't consider FOOM to be a realistic scenario, and this is just more evidence. Individual researchers being made much more productive does not immediately translate into model intelligence; any real-world endeavour has dozens of bottlenecks (like training compute limits, here) that you can't just outsmart. It's similar to the popular visions of moon cities from the 1960s. Our imaginations regarding rapid technological progress always elide the difficulty of actually implementing it.
16:20 More safety concerns: Apparently it's still pretty vulnerable to an attack known as "prefilling", where you make it look like it's in the middle of a conversation where it's already misbehaved. This kind of makes sense to me - after all, no matter how much reinforcement learning you do, it is fundamentally a model designed to continue text, so if you want it to change course in the middle of a conversation, you're trying to override its most basic functions. If you're just using the model through the company's site, they can of course clearly separate their prompt from the user's input, but this might mean they'll have to limit unrestricted prompt-free access. And in some scenarios Little Bobby Prefilling might become a thing.
17:04 As they get smarter, it's getting harder and harder to run alignment testing on models without them knowing they're in an artificial scenario. Interestingly, though, since Anthropic has done a lot of work on introspection, they can actually artificially lower the weights for "I'm in a test", forcibly tricking the model. Like the way that we can turn image recognizers into image generators, this feels like another unintuitive consequence of running an intelligent mind as a program. We literally have the power to mind-control it, and I bet we'll get better at this. (This will be very unethical if AI develops consciousness - fortunately I'm quite confident LLMs don't qualify, but unfortunately I don't think we'll stop doing this even if AI does cross that threshold. AI welfare is something I'm genuinely worried about for the future.)
20:30 So-called "hallucinations" are of course still happening, and I still suspect this is something that we'll never truly defeat, again because of how LLMs work. You don't complete the sentence "The answer is" with "oh wait never mind I don't know". Models might get smart enough to know the answer to most of the things we ask them, which will help, but getting them to precommit to not knowing something (before they begin with the bullshit and can't back out) is an uphill battle.

Context

SnapDragon 3d ago · Edited 3d ago

Unfortunately, while absolutely true, this is a tough message for politicians to sell. (There's a reason we haven't had another Reagan or Thatcher.) People take it for granted that they have all this stuff that barely even existed 50 years ago, as if this is just the way the arrow of time works. It's a lot easier to promise gibs to people who see that their neighbour has something they don't.

Context

SnapDragon 5d ago

But man, it turns out somebody still has to do the hard work of keeping civilization turning so we can keep the lights on until we can finish the silicon god (or the false idol). Those data centers and nuclear plants won't build themselves. Yet.

I have to admit, I have no idea how society is still supporting itself right now. Almost everyone I know is intentionally not working, including myself (got burned out and retired 6 months ago). When I can make crazy amounts of money in tech, then sit back for the rest of my life and have people deliver DoorDash meals to me ... well, who's doing the actual work holding everything up? Some dedicated cadre of 10x engineers?

Sure wish I had the social skills to implement your gratuitous-sex suggestion, though. Not that I think we're in a true singularity - I expect the world to look pretty different in 10 years, but not an unrecognizable ASI dys/utopia. So maybe put the heroin away for now...

Context

SnapDragon 5d ago

I mean, yeah, he's failing (badly) at using a tool that the rest of us are successfully using. And he thinks that this is some sort of flex. It's not worth engaging with him. We're going to keep getting better at using AI, and the serious programmers who aren't just trying to pwn "AI bros" will figure out its advantages and disadvantages, and successfully integrate it with their work. (Or get replaced, if line keeps going up, but the jury's still out on that.)

Context

Culture War Roundup for the week of March 30, 2026

SnapDragon 11d ago · Edited 11d ago

There's a fair bit of fudging, to be sure. Things like running your own model with different prompts or parameters, or just training to the test. But there's a limit. Since the actual ARC-AGI-3 test is not published, the only way companies could really "cheat" would be to sniff the data that's being fed into the models by the testers. While technically possible, that's pretty much Theranos-level fraud; I don't really suspect any AI company of doing this.

EDIT: Oops, I should have clicked the link. The 36% result was on the publicly available data, so it's not really an "official" result. For the reasons @sarker said, I still think it's fine, but it's not quite as bulletproof as I thought.

Context

Culture War Roundup for the week of March 30, 2026

SnapDragon 11d ago

You're assuming he wants a good picture of the capabilities of AI agents. I get the strong impression from the sneering tone of the original post that he wanted to do just enough with a model that he could claim to have pwned the "AI-bros".

Context

Culture War Roundup for the week of March 30, 2026

SnapDragon 12d ago · Edited 12d ago

Indeed, there's almost nothing scientific about the scoring system of ARC-AGI-3; the test itself is kinda neat, and still highlights something that smart humans do (somewhat) better than the best LLMs, but it's dropped any pretense at being an actual measure of "general intelligence", and frankly they deserve to be ridiculed for the sensationalist scores.

Why is completion speed the main factor? Why is the difference squared? Speedruns are not how we define intelligence. If the squirrel in your backyard can solve sudokus, but a top-10th-percentile-of-self-selected-sudoku-solvers human can do it faster, you don't laugh and say "ha ha, this squirrel is so dumb". Also note that the test cuts the model off if it takes 5x longer than the smart human, and later questions build on earlier ones, so if a model goes slowly once it's handicapped for the rest of the test. (Again, this is probably completely intentional, to help deflate scores further.) They used a majority-of-self-selected-humans-can-solve-this metric for puzzle inclusion but not for the scoring. Why? Pure showmanship.

I suspect that average humans who take the test would probably also get a very low score! The old tests and metrics (including ARC-AGI-2) were useful because they showed something that humans genuinely find easy, but LLMs fail at. Those metrics have almost reached saturation, so I guess now we're switching to puzzles that some humans can solve but LLMs ... uh ... solve a bit slower. Ok?

But hey, the "0.5%" number does help low-information AI skeptics like OP point and laugh, so it's another "win" for AI journalism.

Context

SnapDragon 15d ago

You're not wrong. I just don't think there are many good reasons to truly hate Israel, as it's a democratic country that generally respects human rights, in direct contrast to all of its hostile neighbours. (Like Bill Maher says, "one side is accused of genocide but doesn't do it, the other side actually would love to do it.") You can disagree with its politics, and hold it accountable when it crosses over the line (which it certainly has done occasionally - being constantly at war sucks). But I don't think the anti-Israel posters here are capable of that level of restraint - I've just seen too many barely-filtered rants about how the US is being controlled by their evil Israeli mind-control overlords.

-4

Context

SnapDragon 17d ago

I have my doubts, but you make a good point. A lot of the other emergent capabilities have been quite surprising, so there's no guarantee that this is out of the question, either.

Context

SnapDragon 17d ago

I don't think LLMs can generate meaningful human-like feedback of what it feels like to use the software. They just don't see the UI in the way that humans do. And it's not clear that increasing their capabilities can ever fix this.

Still, I do expect that they'll get better and better at iterating quickly and nondestructively based on your feedback, so while it won't be a fully automated dev cycle, I wouldn't be surprised if bespoke AI software replaces giant professional products eventually.

Context

SnapDragon 19d ago

The FDA, obviously...?

Context

SnapDragon 19d ago

Uh, ok, as long as you weren't unlucky enough to be outside the country, being subject to a mandatory 14 day quarantine when you return. And the border was closed to non-citizens for 20 months. Australia overreacted more than almost any other country in the world.

Context

SnapDragon 20d ago

Regarding challenge trials, 1Day Sooner came into being as a result of our clear failure here. COVID was a ridiculously good candidate for challenge trials: a disease that spreads quickly, so every day matters, and which is dangerous to one segment of the population but relatively harmless to everyone else. Our global failure here doesn't speak well for our prospects if a genuinely dangerous plague comes along. (Imagine if the disease had a 30% fatality rate to everyone. Challenge trials would be even more important, and a lot harder to justify ethically.)

I guess the most optimistic take is that if a real threat to society comes along (i.e. a plague which doesn't mostly just replace the "cause of death" for unhealthy seniors), we might actually be spurred to take appropriate measures. It's "only" the threat of creeping totalitarianism which we utterly failed at, enthusiastically cheering on lockdowns and unpersoning anybody who said "uh, wait a minute".

Context

SnapDragon 23d ago

I guess it's entirely possible that was part of the reason! I actually kinda liked Paimon's old voice, but I know I'm in the minority. The new actress is doing quite a good job - the voice isn't too different, but definitely less grating.

Context

SnapDragon 23d ago

Can't speak for the public in general, but that is absolutely why I got into Genshin Impact. (Which is Chinese, so technically not anime, but it's absolutely free of woke BS and performative virtue signaling.) It's so refreshing playing a game (or watching a show or movie) where I don't immediately know who the bad guys are because they're white and male. And where girls are allowed to look sexy, and heterosexual relationships are allowed to exist.

Interestingly, the English voice actress for Paimon (who is the most important character in the game, basically voicing 50% of the lines) actually was a woke lunatic, complete with performative "neurodivergence", an online persecution complex, and claims of being "non-binary". Finally HoYoverse had enough, and 4.5 years into the game's release, they actually replaced her with a proper professional actress. I couldn't imagine a Western studio doing that - if anything, they'd applaud her "bravery" and try to get her on staff permanently to fill out their quotas.

Context

SnapDragon 23d ago

Yeah, we have the right to a trial for a reason. It's kind of stupid to cancel people in the court of public opinion for dubious "unreported" crimes that are decades old. But hey, at least Chavez is dead and doesn't care any more, unlike when that happened to Kavanagh.

Context