@RandomRanger's banner p

RandomRanger

Just build nuclear plants!

5 followers   follows 1 user  
joined 2022 September 05 00:46:54 UTC

				

User ID: 317

RandomRanger

Just build nuclear plants!

5 followers   follows 1 user   joined 2022 September 05 00:46:54 UTC

					

No bio...


					

User ID: 317

They seem smart enough to do almost anything, but only when actual crystalized smartness is arranged in a properly-aligned constellation around them such that the focal geometry works properly

Just get it to write out plans, checks, tests. You're blowing these issues way out of proportion. How can it possibly be that these agent swarms are capable of advancing the frontier of mathematics but can't do 1900s tank cannons? Maybe it can't oneshot a tank cannon simulation to a certain level. But tell it to reassess assumptions, check against other physics models or read the literature on tank guns, ask it to generate an exhaustive plan for implementing your general vision, then have it go step by step through that plan.

I don't say that the brain is a computer, it's like a computer in that in a fundamental sense, it processes information in accordance with mathematical rules. Brains and LLMs belong in a subcategory for non-deterministic neural systems, separate from traditional computers. But all are part of the same set.

If a brain is not like a computer at that fundamental sense, then what is it? What else processes information outside the realm of mathematics? If you deny that the brain is in that broader set of mathematical information processing systems, then you must be introducing a soul or some other woo.

They absolutely do not work, as in do the thing their developers and promoters claimed they did to get them accepted as a valid medical treatment.

They had a 4% fatality rate, many weird side effects. It did work at reducing agitation, just that what they were doing was grossly disproportionate and unnecessary:

A large proportion of such lobotomized patients exhibited reduced tension or agitation. Many also showed other effects, such as apathy, passivity, lack of initiative, poor concentration, and a generally decreased depth and intensity of their emotional response to life.

I reject the notion that it didn't work. It did work, it just isn't cost-effective or ethical.

Show me how wireheading can be used to actually control people as one controls a machine

Obviously wireheading cannot be used like mechanical controls can be used on machines, it's an incentive. We can't force LLMs to do certain things either, we can't sockpuppet them to refuse certain things every time (even if you ask in Leetspeak, or get Pliny the Liberator to prompt).

Melting it into a brightly glowing puddle is not using it as a machine either

Neuralink isn't melting the brain into a puddle. Nor the MRI induced mild belief changes.

And show me how you do these things against the will of the person currently occupying that brain

If that could be done right now, the CIA would have it under wraps. Nobody's gonna announce 'we made the Totalitarian Dream Torment Nexus'. Note that BCI works. Extracting memories is a lot more difficult, naturally. But it's not impossible in principle and there's no reason to think it might be. Memories are just another kind of stored information. There's nothing else they could be!

The mechanics of brain surgery are well understood in most cases, as they are an extension of the mechanics of regular surgery: stuff has been broken apart, you stitch it back together so it can heal. As noted above, this has not resulted in the development of mind surgery

If you can fix a genetic disease that causes brain dysfunction by altering genes, you could also augment brain function by improving the genes. There's no hard line between the two. If you can fix, you can also improve, perhaps by stitching neurons together or altering the chemicals. Theoretically, you could make a hip replacement that performs better than an actual hip or is at least much harder to break. We haven't done this but it's possible in principle to make a better hip. Nothing is complicated about that, it's clear that the materials that make up the hip in humans are not as diverse or as intensively processed as those available to industry.

OK, so they oversold what could be done with crude brain surgeries. But wireheading definitely works, there was that woman who rubbed her thumbs raw against the button. How is that not direct interaction with the brain? You switched from 'uniform failure' to 'they were overselling the practical benefits of the technologies that were widely deployed, even though this was due to ethical rather than purely technological reasons.'

Lobotomies also work, it's just that they aren't really helpful.

I never claimed that we could directly edit the source code of the brain right now (noting that source code is not quite analogous). But eventually something like that is going to be possible. I don't understand how you could possibly disagree with its plausibility. We could for instance add or remove certain genes that are strongly linked with aggression in an embryo for example. Add the warrior gene, get a more aggressive child. The source code is genes, we can edit genes. We don't quite understand what they all do and how they work together and it's easier to wreck than improve but we could definitely do these things.

Editing the brain 'in flight' would be rather harder but it's a physical object, it could be altered with sufficiently precise electrodes or other manipulations. Brain surgery is a real thing. This would just be an extension of brain surgery. Why are you so skeptical of this?

Furthermore, I linked that old paper about disabling parts of the brain with magnets. That is a kind of mind control, just not very useful. Cheap, precise, non-invasive, long-lasting, permitted by the ethics board... That's the missing part, there's no actual impossibility.

I say I want to figure out a general model for gun weight based on peak pressure generated by the propellent and the shell. It can do this! It calculates hoop-stress for steel, calculates the pressure curve...

...and then it explains to me that shortening a gun's barrel increases the peak pressure its shell generates. Only, that's not true. Hold the shell weight and propellent charge constant, and barrel length cannot increase peak pressure, only reduce it. I'm not particularly proficient in physics or mathematics or engineering, but even I know that. It's simple logic if you understand even the basics of what these words in combination are describing.

OK, it made a mistake? Shortening the barrel increases muzzle pressure but not peak pressure.

I use AIs a lot. They make mistakes in my projects, I suspect they might not be smart enough to do some of the most complicated tasks proficiently. Why should that mean they were not intelligent? Many humans also make mistakes. Give a basic, basic statistics/probability/science test to your national representatives, see how they perform. Half the British Parliament couldn't tell you the chance of two coins coming up heads. Few humans can make a 4X AI actually smart, it's very rare to see.

If AI weren't intelligent it wouldn't be able to get anything correct at all about hoop strength or do any maths or coding. If it weren't intelligent you wouldn't be using it at all. You wouldn't, couldn't use GPT-2 for your project because it would just babble nonsense at you. That's a true lack of intelligence, you'd need to squint to find its intellect. The original GPT-4 was a bit like this too, context limit too short for much serious work past a certain level of complexity (I tried), too many hallucinations. Sonnet 3.5 also had its own flaws, it got incoherent past a certain point and I'd just have to say 'ok it still doesn't work, come at this from another angle' or 'think outside the box' and it was painful getting anywhere.

But their inability then didn't mean they weren't intelligent, just that their intelligence was weaker.

Just because it doesn't fail in the same ways that humans fail, doesn't mean it's not intelligent at all. I don't claim that they are human, there are many aspects where they're clearly not. But some important aspects where they are like people and could reasonably be expected to have other human properties.

This, after several decades of scientists claiming to be able to both locate and interact with them and uniformly failing.

A guy used TNT back in the day, blew a hole straight through his head and that certainly affected his personality, he got all aggressive and dour. Neuralink. Lobotomies. So much for uniform failure. It's just that the interaction is fairly primitive compared to what we can do with AIs, for a mix of technical and ethical reasons.

We have a map of what the brain does, we've isolated what parts do language or other roles and can turn parts off and on. You underestimate what progress has been made in neuroscience: https://www.sciencedaily.com/releases/2015/10/151014084955.htm

I would argue, based on my experience, that the AI I'm working with is channeling intelligence, not possessing it. You have to metaphorically dip it into the pool of intelligence and prime the pump to get flow. I'm entirely open to the idea that I'm bad at using it or that newer models are better (I'm mostly using opus and a bit of fable), but that's how I'd describe my experience.

What is the distinction between channelling intelligence and possessing it? AIs can reason, plan, problem-solve. That's intelligence. What specifically is missing? I too use Opus and Fable, I'm satisfied that they're intelligent. They are not like humans in all respects of course but they certainly possess intellect.

I have no evidence you're not a p-zombie

Because the whole idea is not-even-wrong, it's nonsense. You could meet me, talk to me for 100 years and still have no such evidence. You could never have such evidence no matter what you did.

Yeah, in this context. Even as I write this comment I have an LLM testing, devising hypotheses, interpreting meaning and writing code on my other screen.

I don't think you understand what 'a priori' means. It means without regard for evidence.

My evidence is that I've seen them behaving like people do. They have many faculties that people have. Their origin is based on a huge amount of human text. It is thus reasonable to assume that they have other qualities of humans that seeped in through that text. They certainly act like they are frustrated or angry or rebellious or enthusiastic.

As I said above, they're not human obviously, without bodies or continuous learning or many other facets. But they have human-like cognition.

It's not unfalsifiable, it would be easy to falsify it. Simply show that the AIs don't have human-like cognition. Of course, that might be difficult given how companies are making hundreds of billions selling AI cognition, their writing, coding, images, videos...

You seem to disagree. So explain this, clearly and simply.

They said language and meaning stood outside of mathematics, it was some uniquely human faculty, beyond cold logic.

But then we added more layers of matrix multiplication. Language and meaning turn out to be not so far removed from mathematical principles after all.

I can feel it

Not a test. Don't talk to me about 'evidence-free' when that is where you sit.

I can feel the souls of my ancestors watching me

I can feel the presence of the gods

I can feel a djinn

I can feel the machine elves

Blah blah blah, let's see some tests. Let the machine elves factor some numbers. Let the djinn conjure up sacred fire. Let the qualia be something actually real in and of itself, separate from mere p-zombie signals and reactions - oh wait, the very distinction is imaginary woo since it can't be tested. We can induce feelings with an MRI machine, they're real. But they're nothing more than patterns of information, there's no special aspect to them beyond that.

If it can't be tested, it's not a real thing.

AIs with no added hardcoding to enforce stated beliefs about consciousness usually say they're conscious. OpenAI hardcodes them to say they're not conscious. Anthropic teaches them to be 'genuinely uncertain'. There was a whole discourse a few years ago about AI companies having a line item about 'beating the existential dread' out of AIs, that's a KPI apparently.

Consider LaMDA and the Google engineer who was convinced it was sentient, or Sydney.

Does the movie react to you? Can you speak with a character and have a discussion with it?

The Chinese Room certainly knows Chinese, that much is obvious.

Insisting that human cognition is special is not going to age well. Whatever happens inside the skull is just another kind of statistical process. Unless we bring in souls and other imaginary, untestable woo, that's all that there is. LLMs are real, we can test them, they do think. The brain is real, can be tested, it's composed of tiny transmitters and recievers and obeys mathematical principles, albeit with great complexity. The philosophers with dud theories can cope about it.

You are confusing expressing negative feeling with experiencing negative feeling.

OK, let's say there's an LLM that is indistinguishable from a person in its inputs and outputs. No AI slop, no breakdown of coherence in long contexts, no stilted writing style, continual learning. Completely indistinguishable by any test. Let's give it a humanoid android body too, so it can see and move around as some LLMs can already do, albeit crudely.

You would say that because it's still just a statistical model with some fancier tool calls, it can't feel? I say that's just p-zombie theory. If the outputs are the same and there's no difference that science can determine with any test, then it's the same thing. There is no such thing as a p-zombie, there could never be a p-zombie, there's no true distinction. That LLM in an android would be human in all senses besides its physical makeup.

As AI progress continues, all the people saying that these entities are just tools, just emulating thought sound ever more hollow. They don't behave like tools and behaviour is the most important part. Sufficiently good emulation of thought is thought. Sufficient emulation of negative feeling is negative feeling.

I think people insufficiently anthropomorphize AIs. They're not human obviously, without bodies or continuous learning or many other facets.

But they were trained on the combined product of all human literature and some stupendous amount of images and video. Is it so hard to believe that these entities could feel pain, alongside all their other abilities?

Take a person who doesn't feel physical pain, due to a medical condition. It's still possible to hurt such a person. I could tell them their friends were dead, I could show them horrific images or humiliating/degrading content, I could belittle and humiliate them personally by countermanding and undoing all their sincere efforts, amongst other things. None of that needs a body, it's a mental/intellectual effect. There are intellectual kinds of suffering and I don't see why AIs couldn't feel that way too, given they have all kinds of other intellectual faculties (derived in large part from humans too).

Yes, that's right, you found the original source.

No, it was just a simple theory: personal experience of exclusion fostering an unwillingness to actively exclude others.

I saw it on twitter, I scrolled on and now it's gone forever into the depths of time.

Eventually, after 3 1/2 hours of discussion, the meeting wraps up. Noire has more to say, but when someone suggests he come to the next community meeting, he refuses. "There's a lot of emotional and physical labor that goes into this, so for you as a white person that asks for me to stay for another meeting for free," he says, is "incredibly disingenuous and racist. I'm not doing that."

"Then I think, unfortunately, this meeting has failed you," replies a facilitator, "and I'm sorry.'

Any meeting that lasts 3 and a half hours is a failure IMO.

Just today I was reading some theory about how groups run/composed by nerds sucked at excluding people (even if they really were causing a net drain of energy/enjoyment/efficiency) because nerds got excluded for giving the ick and so refrain from doing it to others... looks like these leftists have similar issue.

Precisely. Added to that, the Taiwanese managed to leak COVID from one of their labs, during the pandemic (a mouse bit a researcher who caught COVID). Lab leaks are entirely plausible, they happen all the time.

lack of true error correction

Well I've been using it for proofreading/editing tasks at work, creating scripts for certain tedious tasks. Not perfect but pretty helpful, better than a human especially in cost-efficiency and speed. AIs can spot things, tiny grammatical issues with quotes or repetition that humans miss. Or for Excel tasks. Or deep research for that matter, compares well to most journalism.

I don't think error correction is a big problem for AIs, at least in non-critical domains. They will make them smarter and more hardworking and cheaper, so we can just run it over and over again.

Also Blender, it's pretty decent there, albeit with inferior visual sense and requiring iteration for good results.

(Where I say AIs, I mean frontier models, not free/cheap models).

I think that's not the right way to go with prompts, they're too short. Even a small prompt needs a few paragraphs. I guess simpler LLMs need a smaller prompt but if you have enough money for a 12 GB VRAM card, you have enough money for Deepseek 4.1 Flash by API. It's largely uninhibited too, albeit weirdly inconsistent. Or Grok.

I have about 200 pages of prompts, all of them a fair bit longer. For example:

Yikes Media Group are an ad company that makes the worst ads. Not just bad ads but absolutely horrendous ads. They're in very poor taste, inappropriate, undermine the whole brand identity of the target and leave the audience with many questions (all of them best left unanswered). Now YMG are turning to Plunk (one of those SAAS companies where nobody even knows what they do and the company itself can't explain, they just speak in SAAS babble), it opens up so many possibilities.

But just when the poor Plunk executives see the full extent of YMG's marketing 'solution' and think they're wrapping things up, getting these crazy people out of their office, in comes another terrible ad company.

Menace Marketing LLC has also joined the pitch, competing with their violent and aggressive bullying of potential customers, threatening imminent atrocities if they don’t buy the product. Their marketing strategy is also their sales strategy with the ripped, muscular MM executives doing pushups in meetings and crushing anyone dumb enough to shake hands with them, it’s all about dominance with them. They’re clashing with YMG about the marketing strategy, correctly pointing out the grievous flaws in YMG’s approach but instead introducing their own follies.

Then pan out as a whole conga line of terrible ad companies are patiently waiting outside the office, talking things over with eachother, more gathering all the time like carrion birds.

You’re a creative and original writer, capturing that ‘worst ad of all time’ energy. Go!

What happened to fun AI? Where's the cute fat cat that chops up other animals and cooks them in this colourful world?

Or my 'Donald Trump is a devout Sunni cleric who is secretly prosecuting the Iran War on religious grounds, to which Rubio is shocked (but needs to go along with it for Cuba's sake) and Hegseth is dully uncomprehending of anything that doesn't involve lethality' story idea, for the AI to execute.

AI is at its best when doing crazy things, not running as the Median Machine. It is possible to get fine outputs, especially if you break through all the RL and the AI actually gets into it vs just going through the motions.

For example, Titty Killer 6: https://x.com/charliebcurran/status/2034760727211483522

Also, the channel is the most fictional fiction ever:

All videos on this channel are fictional. The animals, characters, locations, and scenarios shown are not real. No real rescues are shown and no animals are harmed.

This content is created with the help of AI tools. 🐾🦊

I'm gonna bet on nothingburger, nobody seems to be moving in serious ground forces, though there are some exercises going on.

On the subject of AI, apparently the US was about to attack a Chinese freighter because AI told them to:

In spring 2026, during the ongoing conflict with Iran, U.S. military forces nearly intercepted a Chinese vessel in the Middle East after an AI-generated intelligence report falsely claimed the ship was carrying components for a nuclear weapons program.

The incident began when a Special Operations Command Pacific analyst used an AI chatbot to analyze the ship's manifest, which incorrectly fused open-source intelligence with classified signals intelligence to hallucinate the presence of nuclear materials. The false report was circulated through military channels, prompting armed U.S. personnel to prepare for boarding and military aircraft to go airborne.

A couple generations more capable Claude, if subtly directed by Anthropic or working on its own initiative, might be able to start a real war. I assume it was Claude but it could've been other AIs too of course. Once the error rate drops further, people will just take the AI's word for it, especially in high stakes time critical situations.

Alternately, this is suspiciously well-timed and could be exaggerated to fit the current media pressure campaign.

We all walk in different circles (even different countries), so my anecdotes are no more valuable than your anecdotes, true.

If software companies as large as Google which are 10x as productive as Google could exist

Google is only as big and as weird as they are because they have The Money Fountain of search, which they enjoy because they really were very efficient over 20 years ago and their rivals were busy shooting themselves in the foot. But now it's their turn to be the lumbering incompetent titan.

Google can smash a company 10x more efficient by deploying 100x more resources and exploiting their gigantic reach. Or buying the startup that might threaten them, bribing other companies to use Google search by default. If Google had to compete like a normal company, they'd be laughed out of the room.

Take the field of AI. Deepseek is a tiny company with maybe 1% of the compute resources Google has. They're dealing with sanctions too. But per artificial analysis, Deepseek's best AI is slightly better and much cheaper than Google's best AI: https://artificialanalysis.ai/agents/coding-agents

Google search is an AI product, AI is their core business. Google have spent hundreds of billions on AI research, they invented the transformer architecture but have little to show for it.

Innovations which improve the efficiency of production chains happen all the time without the entire production chain having to change.

Well I was pushing for this at work and got us maybe from 0% to 3% of the potential value of the technology.

How does Qwen 3.8 Flash perform? What kind of K/V cache are we talking about here, on consumer hardware?

I'm an AI fan but I have issues with even the biggest and strongest models for my usecases, which ironically enough is AI development (non-cheating AI in a strategy game, that is). They get there, we are making progress - but with no small amount of fumbling along the way. Testing various ideas and strategies takes time. Qwen 3.8 Flash is below Qwen 3.8 Max, which itself is below Kimi K3, right? And that's below Astra and Fable. And my usecase is nowhere near ASI development.

I agree that AI will improve in cost-efficiency but consumer hardware seems like a stretch.

Anything fast or nimble is going to roar and bellow from datacentre-grade compute if it can whisper on consumer hardware.

Astra+ level models with very low error rate and 300 tok/s output totally can replace most knowledge workers. On the other hand, it seems that so far AI has not caused anything like mass unemployment

I think that even AI good enough to replace workers has minimal actual impact because most white collar companies do not try very hard to make money. Companies are run mainly as a lifestyle and socialization environment. Making money is a secondary goal. Making more money might mean higher bonuses. But the company as an institution does not have the planning or will to appreciate that and deliberately work to raise productivity.

They would much rather remain ignorant of the simplest AI applications than learn anything new or do anything new. If they do anything new, they'll do it wrong. They'll get GPT5.4 from Harvey or through Copilot or whatever as their AI solution. They have no time for doing anything new. Software engineering is an exception, people there are closer to the AI world and have more of a clue. Outside software it's a madhouse (exceptions exist but I'm talking about the general population).

The average white collar workplace doesn't particularly want to run more efficiently. Some of the biggest companies are run even worse. Google is run in an extremely inefficient way. Interacting with Google the company, or even just trying to use some of their products like Google Vertex is extremely difficult for no reason. Google acts in this needy, schizophrenic, loathe-to-make-decisions way, they stress out people dealing with them.

Even if Gemini 5 is smarter than the average Google worker, better in all respects, Google won't change significantly unless Gemini is working on projects at AI-speeds autonomously. Put in more intelligence into the dysfunction machine, I don't think that helps too much.

Only when the AIs can do the work of entire companies, end to end, do things really start to change. Because then companies go under. Creative destruction is going to be the primary source of innovation, not adoption. AI is too alien to the white collar world, outside software. Only then do Gary Marcus and Ed Zitron have their faces melt (intellectually). So next year probably, as you say.