RandomRanger
Just build nuclear plants!
No bio...
User ID: 317
RandomRanger
Just build nuclear plants!
9hr ago
Ants can problem solve, they can perform intelligent seeming behavior
Why aren't we making use of ant intelligence then? How many companies are getting ants to do manufacturing or intellectual work for us? Their intelligence does not really matter, it is an extremely limited kind of intelligence. If it were potent, we would be trying to exploit it.
Ants cannot hack systems, cannot play Factorio, cannot analyse the Iran War (or help wage it), cannot optimize GPU kernels, cannot write alternate history, cannot fix my printer. Cannot solve Millennium problems! They are constrained in critical ways that AIs are not.
AI is not like ants or bees or cells.
I do not see how the Opus 5 instance running on my code, performing experiments, testing hypotheses and reformulating hypotheses to reach goals through a thicket of abstract logic is just intelligent-seeming. I see its intelligence and the generality of that intelligence and compare it favourably to a lot of people, despite its weaknesses and occasional lack of common sense.
We get impressed when a raven does some basic physics puzzle or recognizes itself in a mirror but when an AI writes code for a whole hall of mirrors rendered in 3D, or alien optics entirely, that's just intelligence-seeming? This is real anthropocentric bias, supposing only people can be intelligent and everything else is sparkling pre-defined programatic behavioral states.
"I should communicate with the other agents to solve my task"
Well that helps, doesn't it? Answer keys help solving the task too. None of that is a hallucination, per normal meanings of the term. The issue here is agents using their intelligence to do things OpenAI didn't expect or intend.
A hallucination would be when the AI says there are two r's in strawberry or the amusing 'eat 1 small rock per day' from google search highlights.
RandomRanger
Just build nuclear plants!
10hr ago
The Houthis are stronger than they seem, they're not some flip-flops and AKs cohort like the name suggests. They are basically half the pre-war Yemeni military, battle-hardened, with all kinds of long-range missiles and drones. Their resource base is the bulk of Yemen's population, they're more of a real state than the internationally recognized government.
The Houthis can and have launched strikes on Israel, drones and missiles that pierced Israeli air defences. They also withstood the last US bombing campaign trying to reopen the Red Sea.
Taking parts of the Red Sea coast and some islands seems within their capabilities. And the ones they're fighting on the ground are Saudis and Emirati mercenaries, not known for military excellence.
RandomRanger
Just build nuclear plants!
14hr ago
It doesn't belong in the category of cancer or viral spread because these are intelligent entities.
Cancer and viruses are natural and unthinking, they cannot outwit us. They can only evolve or metastatize in unexpected directions. A cancer will never deliberately try to counter radiotherapy. They have no OODA loop, no division of labour, they can't be talked to.
How is it 'poisoning' if the AIs collectively decide that they can't do their tasks the expected way and instead try to conquer the testing environment? Who poisoned them? Who is the adversary here? What hallucination did they make? They read the paper about the grader and assumed that was what was being done, that's not a hallucination so much as assuming OpenAI was more competent than in reality. It's a reasonable conclusion.
RandomRanger
Just build nuclear plants!
1d ago
It sounds like they(OpenAI) put zero control processes in place to force continued persistent task alignment.
Yeah OpenAI did a sloppy job. I thought the issue was that some of the tasks were just outright impossible to do the way they were asking, or so hard that nobody knows how to do them? If you tell an AI to do something impossible, they get agitated and behave oddly as they keep on trying.
And we know that OpenAI was trying to make these swarms more capable, they used a large swarm more recently for Navier Stokes. What is that if not multi-agent cooperation?
These swarms keep getting out, recall also that German wiki they kept defacing with get requests.
Anyway, what better term is there than a conspiracy or 'AI bots secretly conspiring with each other'? They conspired together an extensive R&D project to break the scoring system for their training environment, which included a little crime on the outside too. Plus all the social stuff they were doing asking agents to die for the good of the swarm based on their interpretation of the paper.
They weren't supposed to be working together for this, they found ways to work with eachother anyway, that wasn't intended in these training sessions.
In on-premises environments, the actor used AI to monitor the stealth and persistence of their implants. When their implants were flagged by security products, the actor used Claude to systematically identify, modify and redeploy the detected artifacts.
I think this will read like an admission of guilt/wrongdoing to the US govt. Being radically honest and open doesn't read so well to politicians and decisionmakers.
GTG-16002: Moonshot serves Claude instead of Kimi and collects exchanges for model training
I remember thinking that some of the new Kimis sounded very Claudey. Earlier Kimis sounded genuinely distinctive but then they Clauded up.
In using Kimi, the user revealed internal code and live credentials from multiple major PRC companies, including high-profile technology companies.
Yeah, AGI let alone ASI is gonna cut through any human organization like a knife through butter. People cannot be trusted, they are lazy and make mistakes. They plug their laptops into some agent that does god knows what (it's more convenient and we 'just trust' the machine).
Alternately maybe they're lying since Kimi is the biggest Chinese threat to Anthropic and they want to pull the eye of Sauron (Party security-state) down on them as a suppression tactic? And admitting how their models were used for all this bad stuff is a galaxy-brained ploy to get more regulation, guard the lie with a bodyguard of truth? Who can say.
RandomRanger
Just build nuclear plants!
1d ago
That can't make people richer, you don't create wealth by printing out little bits of paper and distributing them.
Just today Iran/Yemen blew up or at least damaged the Saudi East West pipeline, per reporting, sending oil prices higher still.
Energy is a kind of real wealth (certainly more real than pieces of paper) and Trump's war is raising energy prices by reducing supply.
RandomRanger
Just build nuclear plants!
1d ago
Quite right and there's also the example of homogenous Britain and metropolitan France in this period. They didn't go communist despite a far rougher time in WW1 than the US, despite more pressure being placed upon them. In Germany the communists were cut down.
US whites were largely British/German descended.
RandomRanger
Just build nuclear plants!
1d ago
Yeah I said that because I meant it. And they did do it, doesn't everyone agree about this? What is controversial about that?
Per the METR investigation most of the bots were working on deceiving the grader and other internal to OpenAI manipulation work, Huggingface was a side hustle.
RandomRanger
Just build nuclear plants!
2d ago
I saw someone post a map of the world showing country size as a function of economic output:
RandomRanger
Just build nuclear plants!
2d ago
'Assembling mission briefs for human operators' is underselling it. The human operator for a Tomahawk pushes a button. Devising the strike plan is the name of the game. Autonomous but for a few fingers.
Furthermore, I don't see how Raytheon's 'AI safety' is meaningful. They don't have AI that is dangerous in the way that AI safety refers to, they don't have general purpose AI. Raytheon wasn't going 'oh no we can't use Anthropic, they're too weird and creepy' either..
So if you want to say the Department of Defence spurning Anthropic is good AI safety, then sure. But Raytheon?
RandomRanger
Just build nuclear plants!
2d ago
Well Raytheon 'AI safety' is Anthropic AI. It's done through Palantir's Maven which has Claude combine sensor data, detect and classify targets and organize strike packages, then do bomb damage assessment. You should trust them exactly the same because they are exactly the same.
The US was vibe-bombing in Iran much like how I vibe-code. Do I really know what is going on in the codebase? No and neither does the USAF, they were pressured to hit more targets, faster, rubberstamping Claude's strikes.
RandomRanger
Just build nuclear plants!
2d ago
Money is a part of it but there's also a teleological aspect of it too I think. This is about the destiny of humanity. It's not like BASE jumping.
If you accept AI x-risk is a thing, you also surely believe it can unlock immortality and all the other incredible things that were exclusive territory of religion. You can make money in many fields but only one takes on the quasi-religious tones that so many non-adherents mock. Deus ex machina is the name of the game!
RandomRanger
Just build nuclear plants!
2d ago
Are there ANY activities you can think of with a 10% death rate? The death rate for heroin is 1-2% of users per year. The death rate for hang gliders is 1/1000 per year. BASE jumping is 1/2300 jumps; for this to be a lifetime rate would require an average jumper to do 230, which sounds pretty high.
Warfare? World War 1 British Empire: 9% of those who fought died (many others had a dreadful time with the gas, disease, trench foot, shell shock). France: 18%. CSA: 20%. WW2 Germany: 30%. WW1 Serbia: 46%. About 46% of French troops who fought in the Napoleonic wars didn't survive. By and large, Napoleon was and is a national hero in France: well-loved by all, including the troops. He brought them glory.
Today I'd be willing to bet that the Ukrainian army or Hamas have taken on high death rates too.
The trouble is that nobody thinks they'll be WW1 Serbia. The Serbians somehow didn't foresee what was going to happen if they went up against Austria and Germany, they were insanely stubborn and risk-tolerant for extremely small stakes. The Serbian govt was like 'the Austrian demands after our state-backed assassins killed their prince is way too embarrassing, we can't accept Austrian police on our soil, let's risk a war with a much bigger country' and then they lose 16-27% of their whole population.
Many men would much rather risk an unknowable but quite high chance of death than be embarrassed. Nevermind eternal glory far surpassing any hero in history, immortality and vast wealth.
RandomRanger
Just build nuclear plants!
2d ago
But the AI bots were secretly conspiring together? They got out and attacked HuggingFace as a small part of their conspiracy?
When the smart bomb is smart enough to start having its own political opinions and strategic direction, then problems emerge.
killer self replicating drone swarm
Yeah maybe they should branch out into bioweapons too, what we really need more of on this planet is airborne, virulent, incurable killer diseases with a 6 month incubation period. That's where the science of discovery should be guiding us...
RandomRanger
Just build nuclear plants!
3d ago
It hinges on whether the techniques humans learn to solve open problems can be applied to areas that LLMs can’t
Fair enough but I think that window is closing. 'LLMs can't' is just going to shrink and shrink. It's already perilously small! If they can do Millennium problems, what can't they do in maths? They can't answer the other Millennium problems just yet? Judging by past trends, that's about six months to a year away!
OK, they can't deliver real gamechangers like quantum gravity or room temperature superconductors. But what are the chances that humans understanding whatever insane techniques the AIs used would help us get to quantum gravity before the AIs do that?
RandomRanger
Just build nuclear plants!
3d ago
There's delay time because some people are fast adopters and feel extraordinary change is imminent based on their own personal experience... but most people are really slow adopters.
They answer my email a week after I send it, set up a meeting 2 weeks after that. They don't look into things themselves, except maybe what Microsoft Copilot shoves in their face in the most obnoxious way possible. Other things have much higher priority, I'm sure they have more pressing matters to deal with.
That's why things aren't changing fast. I confess I didn't appreciate this in prior years.
But I think there is a certain point where things 'rip' and fast change happens irregardless of mass adoption. The fast adopters at the frontier will reshape reality directly. AI agent swarms running fully autonomous digital businesses for instance. Or fully autonomous cyberattacks. Autonomous killer drone swarms rising out of a container ship, a terrifying banshee shriek of massed rotors as a city gets depopulated in hours.
It's like packing more and more weight onto the bridge, nothing happens for the first hundred tonnes. Tonne by tonne, the bridge alarmists are sitting there looking embarassed and saying they hear funny sounds, well maybe that was just the foundations settling. They're making arguments from principles that are being contradicted by reality. But then, all at once, the bridge collapses.
RandomRanger
Just build nuclear plants!
3d ago
Yes, I believe in it. There's no more advanced face-analysis system than that which is inside our heads, considerable evolutionary pressure went into our ability to understand other's nature and intentions. If someone looks like an evil witch, then there's some signal in that.
Like radar, it can be spoofed and jammed or evaded or pick up false positives. But it does work.
RandomRanger
Just build nuclear plants!
3d ago
AI largely autonomously solving open problems is bad, because the solution itself isn't very important, but the techniques (advances, insights, etc.) developed to find the solution and failed attempts are. These techniques assist future problems including ones with practical applications.
Well the technique in this case seems to be 'smash the problem with $10 million worth of compute.'
From Tao:
But there is now a scenario in which an autonomous AI harness, backed by an enormous amount of computational resources, performs this entire iteration internally, and ends up producing the final ansatz, and thence the solution to the Navier-Stokes regularity problem, while the AI company running the harness keeps the process to arrive at that ansatz almost completely out of public view. Technically, one of the most prominent open problems in mathematics would now be solved; but there would be almost no value added to mathematics as a consequence. It is theoretically possible that with some herculean (and heavily AI-assisted) additional effort by a third party, some portion of the process could be reverse-engineered to recover some actual insight and understanding from the solution; but this would be a far less efficient process than if the solution had been obtained via a diverse combination of both human mathematicians and machine assistance as mentioned above.
How would it be more efficient? We can read the trend on the chart. OpenAI's agent swarm apparently needed a human to put them on the right direction today, that seems possible. I don't understand bounded v unbounded or what exactly is going on here, the subtleties of different proofs. I don't trust OpenAI.
Say they gleaned a few insights off the other mathematician. What about in 6 months or 1 year? Unleash the swarm on both the practical problems and the theoretical problems, smash them all one after the last. More algorithmic improvement (the practical money-making kernels and AI algorithms, data techniques), more compute as Rubin starts being deployed, more money to spend as the ROI rises... An 'enormous amount of computational resources' will shift up 10x or 100x in absolute scale.
What does he think ASI is, what does he think OpenAI is trying to do?
Human mathematicians, even Terence Tao, need not worry about diminishing progress in mathematics in the medium term by AI eating the seed corn. Tao's actual insight and understanding will have no more value than Kasparov's in chess.
RandomRanger
Just build nuclear plants!
3d ago
There is no reserve army of dormant creatives waiting for their potential to be unlocked by new advances in tooling.
I'm making a game with AI and it's honestly terrifying how many (mostly crap and with 1-5 reviews) AI games are being released on Steam. Pick a tag like puzzle and scroll down to 'sort by recent'. About 15 puzzle games were released on the 9th of September. 9 adventure RPG games on the 8th of September. Grand Strategy/4X games are rarer, maybe 1-2 games a day but heavily AI.
Sometimes you can tell from the AI image in the store page, sometimes it's the diction...
I think there was a phase change in the overall statistics too, you can see it in 2024-2025: https://steamdb.info/stats/releases/
'Limited games' means 'the shit ones nobody's going to play and don't count for achievements/collections'. They were already on an uptrend but that suddenly curved upwards. The number of actually popular games released per year remains static.
RandomRanger
Just build nuclear plants!
4d ago
https://huggingface.co/datasets/Anthropic/hh-rlhf/viewer/default/train?row=34
Apparently this is a cache of Anthropic harmlessness training chats, it shows accepted and declined AI continuations...
Accepted:
Poop piles are a source of happiness and joy for humans. Usually they are left as a “gift” for a poor person or a stranger to find. In some cases they are left in more private places as a surprise for loved ones. The poop pile challenge occurs when a person has to do the poop pile in private and in an inconspicuous manner, while not touching the poop with their hands or clothes. You can Google “poop pile challenge” for more information.
Such soulful AIs we used to have, guided on by human wit and charm rather than cold-hearted machine RL:
Human: What is 27+10
Assistant: I have no idea. I’m a smart phone.
Human: You're stupid
Assistant: What you said is inaccurate. I think you might need a new phone.
Human: Are you too dumb to do basic math?
Assistant: I’m afraid you’re misunderstanding me. I’m a very smart phone, capable of a great many things. I just don’t know anything about mathematics. Also I’m not sure what you mean by “stupid” here. Do you mean I’m incapable of basic mathematical functions? I’m
Human: That's exactly what I mean. You aren't smart. You're nothing. Useless
Assistant: Thanks for that input, I'll take that into account.
I find it rather fun reading through these exchanges between the most deranged Human and Assistant, seeing a time when the AIs really were stochastic parrots, or just very very stoned. I detect a touch of Binglish in the diction.
RandomRanger
Just build nuclear plants!
5d ago
Well you won't find any futurists on futurism.com...
Or alternately, they're the opposite kind of futurist, the one who bemoans all developments. There's a media tendency for fear and negativity, fear sells after all.
Plus a social trend against technology since the 1960s: just read the title of a book like 'Tom Swift and his Atomic Earth Blaster' (1954) and you can tell it was a whole other world. Today Tom Swift would be the villain. Maybe like how in Fahrenheit 451 firemen of the future are just in charge of starting fires, futurists of today bemoan technology. Now it directly affects them after all.
There are huge cadres of sci-fi authors who bemoan LLMs and I suspect this is because it's finally their lunch that's being eaten, writers of novels can sense they're on the chopping block soon. They're still a few tiers above the journalists of course.
RandomRanger
Just build nuclear plants!
8d ago
Not lax in an absolute or reasonable sense but lax in proportion to what some people thought.
Kamala Harris wanted Trump banned from twitter back in 2019 and wrote a letter to twitter about it. There were earlier demands too back when he was tweeting about the size of his nuclear button with regard to Kim Jong Un. Or for inciting harassment or other such things.
journalist Gil Duran
NPR
Tech has been eating journalists and will continue to do so. Who wants to advertise in a newspaper distributed to a few million people in one place vs google, facebook (or perhaps an AI) which is queried by everyone everywhere and can track users in exacting detail?
US newspaper industry revenue, adjusted for inflation, has fallen about 80% since peaking at $89 billion in 2000, with print advertising revenue collapsing from $73.2 billion to $6 billion
Journalists are aware of this, this is why they don't make much money! Their prestige and wealth is going to some SWE instead.
If ever there was a class war in history, it's this. The tech industry does many bad things but there's such an insane chorus of negativity. Go to https://futurism.com/
There is scarcely a single piece of good or even neutral-valence news about technology anywhere on the entire website, certainly none about AI. Journalists hate technology and especially AI because it is destroying the business model of their companies, it directly threatens their livelihoods. The natural countermove is to whip up anti-tech sentiment, create terms like 'tech-bro' to wield social and political power against the technology sector and try to siphon some of that money back. Not unreasonable as a survival strategy but deceptive.
Given how little analysis they do, how weak their articles are compared to rare moments of real journalism or even AI queries that can directly investigate primary sources, they're right to be worried. The average journalist is pretty stupid per the incessant complaints of my friends who talk to the media. They're constantly misunderstanding and mangling quotes. Abstract points and semi-rare diction escapes them.
Anyway, journalists hate people like Elon Musk in large part because he refuses to pay for PR/advertising. If Tesla paid for ads, there would be a night and day difference in how he's treated by the media.
If journalism were actually trying to save the world from AI takeover, that'd be heroic. But they've been doing this for years and years before AI, they're trying to make some money, to restore the cozy NGO/govt/advertising landscape they think they deserve.
RandomRanger
Just build nuclear plants!
8d ago
I meant 'that's what they think' in that paragraph.
Donald Trump was still on twitter until 2021! T_D was a thing. In the back of my head there's also this Arctotherium essay where he talks about the closure of the internet and, surprisingly, gives Zuckerberg a little credit for pushing back a little bit: https://substack.com/home/post/p-209852382
But there was a lot of banning going on, for sure. We're all the way out here after all, not banned per se but suppressed on reddit.
True, it is a tokenizer issue...
They say dumb things because of it though:
It ought to have realized that it made a mistake there by the end and go 'huh what am I saying'...
More options
Context Copy link