aqouta
Friends:
User ID: 75
aqouta
1hr ago
The automated CW threads are a matter of our history, we trace our lineage back to Scott's weekly culture war threads when he had them on SSC before the CW got hot enough that they were pushed first to his subreddit and then off the subreddit to /r/themotte, finally we left reddit with some help from the rdrama.net fork of ruqqus. Rdrama.net probably belongs in your cataloguing of far flung communities. The weekly threads are intentionally where the culture war mainly is and that's the thing people come here for. Sometimes we post to the main wall but for instance your post would probably get more traction as a top level CW thread in the main weekly.
aqouta
1hr ago
Your position upstream was "it's simply not the case that they give the labs asking for them a competitive advantage" and then you asked me to "point to a regulation proposed by any of the labs that would actually hurt the little guys more than the frontier labs? Or at all?" Seems like you can create a competitive advantage without locking down the entire space for yourself.
And you didn't, you refused to do any work so for the conversation to continue I cited the proposed legislation myself which does not hurt the little guys more than the big guys.
As we discussed previously, you can hit the FLOPs requirement to trigger oversight without spending billions; Mistral did it. Now, it is true that Anthropic's proposed rules would also not apply to anyone who made less than $500 million in annual AI-derived revenue or spend more than $1 billion on R&D annually, which doesn't seem to have anything to do with the frontier at all - if someone decided to make an Evil AI in their basement/supervillain lair based on an existing open-sourced design, it wouldn't need to comply with Anthropic's model regulations, even if it exceeded the FLOPs requirement (which some open-source models do).
the openai FLOPs requirement is 10^26 which cuts a bit higher, but if your argument was "actually it should be 10^27" then we can have a conversation. But you don't seem at all interested in engaging with regulation as if it's a valid end at all. You make arguments that generalize to all regulation in all cases is bad because you can make up a story for how this somehow helps incumbent companies.
If I am getting your argument right, then, application layer startups are against regulations less because it might affect them directly (in terms of oversight) and more that it will slow down the progress of the frontier models that they interface with to make their "absolute rancid slop," thus slowing their revenue gain. Is that a good summary?
They're part of the a16z group that broadly opposes all regulation and actively funds pacs to this effect(leading the future). Their opposition to this regulation in particular is over determined. They want the frontier to advance quickly and be distilled by open weight models because it's their belief that every other layer will be a commodity layer and they're the real enduring layer. They have a love hate relationship with the frontier labs. They also oppose all regulation in general because they fear if some AI regulatory organization got some teeth they might come after their slop, which the public widely hates. for fun here are a few examples of a16z funded projects:
Doublespeed Slop social media spammatron 5000
We've built an attention intelligence platform to build and deploy social agents at scale across every demographic and niche. Capturing mindshare on organic social is currently a human labor problem, instead our autonomous creators learn from every interaction, and become more valuable over time driving conversions and engaging with millions 24/7.
Cluely literally cheating as a service
Cluely, a startup that claims to help users “cheat” on job interviews, exams, and sales calls, has raised a $15 million Series A led by Andreessen Horowitz, the company announced on Friday with a video posted on X.
Character.AI this one literally has blood on its hands
In November 2023, 13-year-old Juliana Peralta of Colorado died by suicide after extensive interactions with multiple chatbots on Character.ai. She primarily confided suicidal thoughts and mental health struggles in a chatbot based on the character Hero from the video game Omori, while also engaging in sexually explicit conversations—often initiated by the bots—with others, including those based on characters from children's series such as Harry Potter.
aqouta
3hr ago
Yes, none should probably advance much further, at least in this form, until they can collectively get that under control. If the regulations freeze the frontier then that seems good for the little guys trying to catch up.
aqouta
3hr ago
Then they'd have been exempted from the regulation or have to do a fairly small number of things like submit the model to review before deploying it. It just really don't seem like a huge barrier when you already have to be spending a billion dollars to even qualify for regulation.
aqouta
3hr ago
If you have to run a race where the leader is 5 km ahead with training weights on, and you also have to put on training weights once you get within 1 km, it sounds like something that will severely slow down your ability to catch up. Maybe not for ever, but for long enough to count for the mortal leadership.
I fail to see how this meaningfully is different to trying to run that same race where no one wears safety equipment. And comparing basic safety and verification checks to a 5kg vest is silly. They kind of company that can meaningfully challenge the frontier is like oracle sized or a new startup with industry insiders and several billion dollars in funding. There just really can't be a small guy structurally when you have to bring billions of dollars to the table anyways.
And while "we could all die" would be very unfortunate, the probability is not yet above my personal willing-to-be-Pascal-mugged threshold. Especially given that we will all die in due time unless sufficiently fast progress in medicine makes it otherwise, and frontier labs being mired in regulation sounds like the opposite of fast progress.
Just the diffusion of what we already have will take a lot of time and I think advance medicine quite quickly. And no one is proposing preventing narrow drug research models. This isn't an either or thing. This is trying to avoid going supercritical. We really don't need to roll these dice to get radical advances.
aqouta
4hr ago
Can you even explain how the tangible results give them an advantage? The best story I can come up with is that they're able to comply with the regulations better somehow because they were already big. But it's hard for me to imagine a startup lab that is able to get billions of dollars together for a training run that can't figure out how to follow regulation. And the actual tangible direct effect of the regulation is that they stop pushing so far ahead, which just obviously gives other labs a chance to catch up as a first order effect.
Even if I grant that it gave them some unforeseen or specifically articulated advantage surely "we could all die" is enough reason to at least make an attempt that we can modify later if someone does find some harm later? It's not like anyone is saying these rules should be written in stone and assumed perfect for all time, we're trying to coordinate a slow down and ensure safe practices. This is a red hot industry at the center of discourse, it's not likely that any harms will go unnoticed or discussed.
aqouta
4hr ago
Usually, when a scenario can be explained by "huge corporations conspire to corner the market" I would rather look for evidence that they couldn't do that rather than that they did it. Because my first question is "why wouldn't they?".
Ok so look for it. Did you find any? Do you think it's categorically impossible that driving forward into Recursive self improvement in a race condition could be dangerous? It's very exhausting, you have a motive and are in search for a crime, but not a real search because you don't bother to actually look. Meanwhile we might be driving off a cliff.
aqouta
4hr ago
As per our discussion last month, it seems pretty clear that Anthropic's proposed regulations would hit "little guys" like Mistral, and their regulations impose a burden on small companies, burdens are costly, cost hurts, ergo is seems pretty clear that the regulations would hurt little guys. (This does not necessarily mean that regulations are bad.)
Ok, so Mistral would be required to to submit to the security requirements that the other labs have to submit to. Such as registering with CAISI or some other organization depending on what blueprint you're talking about and submit their models to pre-release testing. How does this lock them out of the frontier exactly? I actually wouldn't be surprised if they already had stricter requirements from the EU given they're based in france.
With that being said, I'm still not clear on your argument. If your position is that the regulations wouldn't hurt the little guys, or would hurt the frontier labs (whom you say they hate) more so than the little guys, improving the competitive position of the little guys, then why are the little guys against the regulations?
They want the labs to push the frontier forward as fast as possible, the labs want to slow down. They also find it annoying when the labs have guardrails at all. There's two things "the little guys" could mean, smaller labs and "application layer" startups. The concern for the smaller labs seems totally hallucinated, the idea that a lab is going to be able to put billions into training runs but can't comply with some oversight is absurd on its face. The real concern is about these "application layer" startups who are shoveling absolute rancid slop everywhere as fast as they can. These slop merchants don't discriminate on what type of regulations are proposed for AI, they oppose it all categorically.
aqouta
4hr ago
They didn't originate from bills, but from my understanding of AI safety discourse, the version of that discourse being presented by the lab leadership, their calls for regulation, their market position, and the general way they've been generating "scary" AI incidents that don't seem organic and do seem to bolster their calls for "regulation".
So a narrative story you've been told that you don't feel a need to ground out in any actual actions taken by any of the people or groups being discussed? They're not intentionally committing felonies, that doesn't make sense. The downsides on that are wildly out of proportion to any imagined upsides.
...how is this not a moat?
How is it a moat? If you're spending less than a billion dollars then you're entirely exempt and if you're spending more than that then you just have to comply with simple things like pre-release testing by a third party that all the frontier labs have to comply with. How could you possibly design a regulatory system that is less moat shaped than that?
aqouta
4hr ago
Do you have any actual evidence of this? The cartel accusation as far as I can source it comes from them asking for an exemption to the Sherman act so that they can work together on standards, which is common among many industries and would not at all form any kind of moat. There's a lot of accusations being thrown around which as far as I can tell are entirely hallucinated.
what is an a16z type person?
Andreessen Horowitz circle, I outline the perspective here.
More generally, I have a fairly embedded prior that the AI companies are in fact actively attempting to secure a regulatory moat, wherein they are the only companies legally permitted to provide AI services. Actions they've taken, particularly the "rogue" hacks, appear to me to fit this model quite well.
Am I being lied to? which bills are supposed to disprove this prior, and why?
From which bills did the priors originate from? Openai and Anthropic have both published blueprint suggestions that explicitly exempt anyone doing training runs below 10^25 FLOP earn less than $500M in AI revenue or spend less than $1B a year on AI R&D. If the worry is that the regulations would create a moat we could actually talk about the regulations and demand they be written such that a moat wouldn't be created. But the people pushing this point have not read any of the proposals and do not care about the moat argument besides its use as a soldier.
They don't want any regulation ever or for things to slow down, more broadly they hate the frontier labs in general. There are a few groups in this category, some are angry that they passed on investing in the labs early on in favor of crypto. Broadly their preferred future looks something like the labs burning the money of other investors to produce powerful models that chinese labs are allowed to distill and then they can run them locally to power their anti-social revenue extracting products. They see themselves as the "application layer" that Huang often talks about. They have very little vision about the future of AI and broadly only care about what makes their low effort wrapper start up more money.
Can you explain the alternative view? Can you point to a regulation proposed by any of the labs that would actually hurt the little guys more than the frontier labs? Or at all?
The vote difference with no actual rebuttal is pretty funny. People you trust are lying to you. You should not trust them. These are lies cooked up by A16z type people because they want to be able to produce their slop gambling porn simulators more cheaply. That's the faction who's lies you're swallowing and you're getting mad at people for pointing it out.
You can look up the proposed bills, it's simply not the case that they give the labs asking for them a competitive advantage. It is not an opinion. It is untrue. It is a lie.
aqouta
1d ago
He also co-wrote big yud's "if anyone builds it everyone dies" book. He's not a big blogging voice but very much in the circle.
aqouta
1d ago
as one of the broad reasons for calling for regulation was to get a legally enforced competitive advantage
This being entirely untrue must have been explained to you before, why spread the lie?
aqouta
1d ago
I'm sorry but "they're fighting these people with insidious methods but they're bad at it" is such a retarded line of attack. These people are running some of the most valuable companies in the world. You have to do better than this.
aqouta
2d ago
There isn't an argument being made. "a contemptable language game" come on man, he was pointing out some amusing parallels. That people who hate him and his movement are blowing it out of all proportion
Christians' moral objections to Aella's activities and to the Yudkowskiites' exploitation of them for tactical advantage
Tactical advantage? What are you talking about dude? You already said this is not to his advantage. This isn't some weird 10 diminsional chess game, they're just abnormal people. They're the type of people who take thought experiments seriously and follow them to their conclusions. Turns out doing that a lot lets you make some pretty surprisingly accurate predictions about how important things like how AI would go. They also take things like liberal sexual ethics to their logical conclusion, which you oppose but isn't really an argument against their other predictions.
I do not accept that the Yudkowskiites or the "Effective Altruism" movement they have parasitized
parasitized? They birthed the movement.
aqouta
3d ago
The context is that there is kind of a civil war in Scott's general sphere of twitter over the AI saftey question. Scott, having a history of being tolerant of religious rightists during the culture wars, has a wide number of Christians within his social web. These Christians and rightists variable side with or against his positions but often remain civil. It wasn't obvious that they were going to take the anti-ai-safety position but it's starting to seem like the worm is turning that way and there are a lot of attacks on EA/Rationalists/Safetiest from the Religious right direction. The subtext is kind of clear "We know we're weird by today's standards and but were you and your people not once weird by today's standards?" He's being accused of doing the meme but scott has engaged earnestly and in good faith the christians frequently enough that I don't think you can accusing him of cynicism here. He does in fact think it's admirable that Jesus forgave prostitutes and preached against a corrupt moral order, in many ways his movement is downstream of christian enlightenment.
aqouta
3d ago
My response to both of your points is the same. Unless you specifically prompt them otherwise the output from frontier models is always very hedged, which is why it half agreed with MKC without being willing to really fall on one side. You're taking a hedge as disagreement, in reality it neither endorses nor disagrees with MK, which is a fine null hypothesis. Where it doesn't hedge and outright states something is false or true it's pretty trustworthy unless you have strong evidence.
Zvi calls this concept bounded mistrust and it applies to all media sources. They can mislead but almost never lie.
aqouta
4d ago
I'm not looking for someone to help finance expensive vacations or get me into exclusive parties, I'm looking for someone who I enjoy spending time with and who will be a good mother, which if anything slightly anti-correlates with a high status career.
Of course earning potential looks bad when you frame all the upsides as being used to purchase disposable hedonistic experiences. Women provide half the genes, my doctor wife's gene combined with my software engineer genes are going to produce an autist that will rule the universe. The extra money can be spent on a home in a great school district where my kids will be among the kids of other people with great genes and highly agentic parents. And despite the caricature in right wing circles successful women are very much still in possession of nurturing instincts.
It all reads like the sour grapes of someone who thinks individuals need bad traits to counterbalance their good traits for it to be "fair", like it's a video game where the classes have been balanced. There can be downsides to pursuing high status to be sure, but having it is nearly a pure upside. Tales of how money is actually a curse are almost all cope.
aqouta
4d ago
I actually think that prediction markets are very good. But my comment is about getting you to acknowledge that LLMs are not a source of evidence.
Frontier models are definitely a form of evidence just not proof. I think it'd be fair to call their position the null hypothesis from which to argue from, not infallible by any means but if you disagree you should have a reason. And the response you got from claude barely disagreed with MK.
aqouta
4d ago
HotS was very fun, I was a GM for many seasons. The thing that made it great but was the same reason it was never very popular was because it's designed around team work.
aqouta
7d ago
An unrealistically absurd amount of the stock valuation was based on xai. It doesn't really work as a directional windmill for how well the rocket business goes. I was disapointed because I wanted exposure to the rocket business but buying 5x as much exposure to xai to get it feels insane.
Yes, I don't think they'd be a moat. It's up in the air, as they're not strongly defined, if they'd net help or hinder catch up, as written I would guess they would help catch up. Slowing the front runners down gives those behind to fast follow at leisure. This is actually a negative in a sense to me because it lets chinese labs catch up relatively and there are world where that is quite bad by my values.
I don't like to make the arguments from incentives on this subject because I don't think they're the right frame of the argument. It's the frame the a16z/accelerationists want to push because then it's a matter of their interests vs anthropic and don't you really care about freedom of speech ect ect? But I don't think the important point here is about who's interests should prevail, even if that's also for contingent circumstantial reasons an argument my side can win against them. Because even if the balanced tipped and from interests alone the labs came out looking worse through this cynical lens I still don't want everyone to die. If entrenching AI control into a handful of companies actually was the only way to keep everyone from dying then we should do that, albeit probably it would be best to nationalize them in that case. I don't think that's the case but I don't want to hinge my defense of regulations on this unstable ground.
More options
Context Copy link