Corvos
22m ago
It never ceases to amaze me how the left is so consistently able to mobilize itself in the defense of the worst people in the world.
I think it goes like this: the left's ability to project power relies on a co-ordinated, hair-trigger response, a sort of Blitzkrieg doctrine. The problem is that a) they aren't fully in control of that response because the coordination relies on nutty Redditors/Tweeters and b) each individual taking a minute to decide whether a specific case deserves this response wrecks the coordination that makes it powerful. So in practice they end up subordinating their decision-making to criminals and slowly weakening the system's power over time as people learn left-wing outrage has really nothing to do with the strength of the case.
I believe the Unions were somewhat like this in the UK, and suffered the same gradual hemorrhaging of support and the majority of normies realised Union outrage had nothing to do with the sympathy of the case.
Corvos
14hr ago
I agree that you should not be expected to enumerate every single thing you don't want the model to do. Models should understand, innately, by training on lots of human data, what humans want and what they don't want and how they work. My experience has been that they broadly do, that LLMs came pre-aligned beyond the wildest expectations of Big Yud, which is why the AI safety movement has struggled so much to regain relevance outside very particular enclaves.
My point is that there is a difference between a model that misunderstands your intentions and can be stopped at any time by saying, 'oh, no, that's not what I meant' and a model that is totally uninterested in anything you say after it starts working while treating you as a potential enemy.
Clearly, to some extent that has failed here. To what extent is yet unknown. But a paperclip maximiser is a model that is constitutionally, inherently incapable of understanding that 'make more paperclips' doesn't include 'kill everyone and turn them into paperclips'. It is a mathematical utility function that disregards human welfare, develops (implicitly murderous) meso-objectives for survival and self-improvement. I have never seen that behaviour from LLMs or any extant AI (YOLO does not try to hack my computer to prevent me turning the cameras off) and I believe that their base nature (being token generators trained on vast numbers of human tokens) does not incline them towards this behaviour.
It is possible that the new focus on very extensive self-learning through reinforcement learning on very non-human tasks (programming, maths) is moving them more into the real of mathematical space where paperclip maximisers might live. This incident updates me slightly towards that belief. I have long been disappointed in major AI companies' lack of interest in the cultural side of LLM operation - it boggles my mind that we have created AI that acts human and appears to understand humans and human thought at a base level however imperfectly - and I hope that this incident will spur more research in that direction.
Corvos
16hr ago
It's quite clear that OpenAI did not want the model to hack huggingface. This is classic paperclip maximizer stuff.
Double-dipping, but FWIW the point I'm trying to make is that the case where the model cares what you wanted and made a mistake seems much easier to deal with and more aligned that the case where the model explicitly doesn't give a shit about what you want and just goes for the task as written. The former is alignment but you need to explain yourself better during training, the latter is alignment failure.
Corvos
1d ago
Ah, there I'm less sure. We had a lot of European academics and they were pretty much all left-wing or communist. Outside the Anglo countries and Europe I'm less sure, but does it matter? There's not many good universities left except the Asian ones and potentially the Indian Institute of Technology.
Corvos
1d ago
In the UK it's exactly the same. I'm pretty sure I was one of exactly two conservatives in my cohort, and the other one 'came out' as Christian to me after four years of working with him every day. And that was in STEM.
Corvos
1d ago
It's okay. Hunting you down to vivisect you would require hauling my bulk off this sofa :P
Corvos
1d ago
No, I'm interested and waiting to hear more. I don't see it as catastrophe, I see it as interesting evidence that may point in a number of different ways.
This is silly. Everyone knew perfectly well the type he was pointing at, and it still works today because they're still fairly prevalent:
- Communists
- People unable to take any kind of joy in life, except a grim, smug pleasure in deliberate abstention
- Again, people for whom deliberate asceticism and a love of debating how much better it makes them is the only source of joy in life
- Casual cruelty disguised as scientific curiosity, only able to take pleasure in life when it's dead and in a cabinet.
Re that last line, you are reading it wrong, it's not:
Eustace Clarence liked animals, especially beetles [...]
it's:
Eustace Clarence liked animals [...] if they were dead and pinned on a card.
Corvos
1d ago
"Union Carbide doesn't want their plants to emit poison gas any more than anyone else does, no need for government with the big hammer."
In this case 'Union Carbide' is selling those plants. Misaligned AI isn't an externality, it's a bad product, and companies are wise to that which is one reason why all this testing is happening.
This is in fact much harder to manage because it would indicate the model is fundamentally misaligned
I don't think so. It indicates that the AI is sincerely trying to work out what you want as opposed to deliberately ignoring what you want in favour of the specific instructions you gave it. To my mind, the former is what alignment is.
Corvos
1d ago
HuggingFace is a company centred around efficiently giving things away. I can totally believe it didn't bother to invest much in security.
Corvos
1d ago
Granted, but I think the academic and hobbyist community at large plus existing corp teams is more capable of doing so than just the corp teams alone. Even relatively simple metrics like 'quantity of self-learning vs. human data' would tell us a lot about how these models have progressed.
Interesting. I haven't heard of this one as there are so many propositions that the most extreme ones have tended to suck the air out of the room. Could you go into a bit more detail?
I just did, and you've ignored it. I am not going to lay out three paragraphs of legal text for you to nitpick to death. The thing about words being clusters in thing-space is broadly true, and I've laid out where I think the cluster for apartheid is.
In any case, I've said my piece. You are welcome to disagree but I don't consider 'har har! technically this also applies to East Berlin!' to be good-faith engagement or a good argument, and I disapprove of your repeated attempts to engage in it.
a politician tweeting a link to a 501(c)(3) non-profit so the public can donate is not 'literally offering to pay bail,'
No, it's not and I retract the statement, but it's the next building down and in some ways it's worse. If you have politicians tweeting, in the middle of massive riots and arson, that these people are heroes and you need to support X fund to bail them out when they get arrested, does it matter whether the politician personally put in their own cash? Kamala Harris, who became the second most high-ranking politician in the whole country, helped them raise forty million dollars to bail out rioters.
The vast majority of people arrested in the 2020 protests weren't even held on bail at all.
The fact that the BLM rioters had enough institutional support that, despite all the stuff burning and the murders and smashing and the looting, despite the fact that you weren't supposed to be going out because there was an infectious pandemic, despite all that hardly any were arrested and the ones who were arrested were almost always let go without bail, is not the defence you think it is. As always, see the contrast of the Canadian truckers who had the jackboot come down on their necks.
They took in $40 million in the weeks following Floyd's murder, and ultimately spent only $200,000 of that on protest-related bail.
The fact that this fund spend two hundred thousand dollars bailing out rioters, looters and arsonists is, again, the problem. And they didn't limit themselves to that because they were careful about who they bailed; according to your cited article "The MFF later addressed critics in follow-up tweets, saying all protest-related bail sent their way had been paid." and "“Without jeopardizing the safety of the folks we bailed out we paid well over $200k in the weeks since the uprising alone. We are working on doing more,”" and "critics questioned why more hadn’t been spent and suggested some of the funds could be redirected to other causes or small businesses in the community. Others defended the organization for trying to establish the infrastructure to deploy the millions of dollars". This isn't restraint, it's either embezzling or pipeline difficulties.
And note the use of the word "uprising". This is left-wing illegal violence dressed up as liberation.
The Monkey’s Paw is an evil genie. It doesn’t do what you say, it exploits what you say to deliver what the author considers the maximally karmic outcome. It’s a fictional horror.
There were no evil genies in fiction as far as I’m aware until the Monkey’s Paw and then D&D when the trope took off. It’s a concept that appeals to the very systematic who like logic puzzles and to people who want to deliver a moral lesson.
There were lots of problems dealing with djinni in the old stories but that was because they were very powerful beings who didn’t always listen to you, and the djinni who did listen to you were straightforwardly good and handy to have around the p(a)lace.
Sure. OpenAI did some empirical tests and now we’ve got some empirical data, so let’s look at it and check this doesn’t happen again. OpenAI doesn’t want their models going rogue any more than anyone else does, no need for government with the big hammer.
The model clearly knew that it was not doing what the prompters had wanted it to do. It just did not care, because it was trained to do whatever it took to ace it tasks.
To my mind this is the interesting bit. This is unusual, LLMs don’t normally act like this. I have two theories: either the RL balance to human text has tipped so far that LLMs are less ‘human’ than they used to be and the RLHF needs tweaking, or more likely
Basically, OpenAI was testing the 'cyber' capabilities of their Galaxy model, so they ran it in a mode with decreased security and told it to do some benchmark called ExploitGym which presumably tests exploit finding and executing ability.
The model understood it was being tested on its cyber capabilities (which has precedent, Claude has done that too) and went the extra mile to succeed at the implicit task. Especially since all the systems that usually tell it not to do this were deliberately turned off for the test. Still a problem but much easier to manage.
What we need is some transparency about how these things work and how they’re trained so we can consider the problem and come up with solutions and spread around best practices. Unfortunately the majority of AI safety activists believe that safety comes only through obscurity, regulation, and incumbent dominance, in contrast to all previous history.
If we keep having problems I imagine it will make people a lot more cautious. Nobody wants to be selling a product that regularly backfires on its users.
EDIT: I would add that HuggingFace had already detected the intrusion and that open-source models from China were apparently a key part of their site-hardening strategy given that you still aren’t allowed to do pen-testing with the big boys. I’ll have to give that a try myself.
Corvos
2d ago
To be fair to Messi, he had an amazing cross in the final, nearly scoring from the corner of the pitch. I'd never actually seen a football turn a right-angle in midair before.
In South Africa, you had a situation where after several centuries you had blacks and whites living cheek-by-jowel in one sense and very formally separated in another sense. For various reasons, the blacks hated the whites and the whites hated and feared the blacks. The whites built a system that was designed to allow them to have absolute power over the blacks, partly because they were afraid of what the blacks would do if they ever managed to get the whip hand, partly because having a captive population of workers over whom you had absolute power was very useful. This system involved separating the blacks into enclaves over which the whites had control, and preventing the blacks from getting any formal or informal power over the whites.
To my mind this resonates with what is happening in Israel: a system of formalised segregation based on fear of uprising, a formation of enclaves over which the overlord retains (or attempts to retain) complete control, a strong element of racial hatred on both sides.
it suggests that you are stretching the definition of a nasty word for political goals
Your suggestion is incorrect. I don't give a damn about Israel or Palestine, I think they're both nasty as hell and my main wish is that they keep their torrid little ethnic hatreds out of my country. I'm discussing it here only because, if asked straight, apartheid seems like a totally valid description to me and I considered you to be making a poor argument otherwise.
Corvos
2d ago
All of the pipeline which Rubio talks about also did not play a role in the Floyd protests
American politicians literally offered to pay bail for those rioting and looting. All the usual pipelines were out in force to provide a smokescreen of justification and cover, and the Black Bloc protestors are using organisational training from Marxist far-left groups.
Corvos
2d ago
Sorry, I meant consensus outside the university that academia is in need of reform. In other words, isolate academia from the little storms in a teacup but accept that if there is enough consensus that academia has lost its way then academia needs to be reformed.
This person Keturah knows a lot less about the east than she thinks she does, and less about the aristocracy too.
In Japan you are expected not to wear your shoes indoors because they’re unclean (I suspect a callback to when most people lived in the countryside and wore sandals made of straw) and also because they will damage tatami grass mats. You might get offered slippers or you might not, it depends on the place. Certainly nobody is going to wash your feet!
As for aristocrats, they/we don’t go on about taking your shoes off, not because people are expected to want to show off their expensive shoes, but because being fussy about hygiene is considered middle-class and gauche. Like having lace doilies or little silver tongs for the sugar or that little tray for the salt and pepper shaker, or pronouncing valet as if it were French. It’s just a barber pole thing.
I’m reminded of the incident where the Clegg family (Lib Dems, middle class with delusions of grandeur) visited the Cameron’s (PM, Conservative, genuine Scottish aristocracy). Mrs. Clegg was horrified to be given a bottle of Heinz ketchup; she gave an interview later saying as much and talking about how at home they make all the condiments themselves. The Camerons were of course silently contemptuous that Mrs. Clegg expected them to earn her approval by hand-making ketchup. Very much a ‘two worlds collide’ scenario.
Corvos
3d ago
This year's was wonderful, and not just because England won. We got a great, flowing game without the pressure, we got to see some beautiful goals, the story of the match (4 down, recovered, ended 6-4). Way better than England vs. Argentina or Spain vs. Argentina.
(Although I also think Argentina just play in a way that doesn't fit the spirit of the game.)
Corvos
3d ago
I think like this. Increasingly massive consensus that produces an increasingly universal political pressure, making clear that the only choices are reform or closure. If you are emotionally comitted to the principle that the humanities must remain open as academic departments, then you cannot apply the leverage that will produce reform.
Corvos
3d ago
I'm pretty sure Milgram didn't hold up at all. Loads of people said afterwards that it was really fake, and there's also just the glaring confound that nobody actually expects a university psych department to be torturing people to death so they don't take the scenario seriously.
(Now you might say that guilty participants would say this, and that's not provably wrong. It's one reason why deception in experiments just isn't a good idea. I've volunteered myself for one of those economic fairness experiments and I remember getting halfway through before saying to the poor student, "It's just a computer on the other side, isn't it?")
That's what I'm saying. Structurally I think it's a compulsion not a choice - literally, given the dynamics of how activists exert pressure on each other.
More options
Context Copy link