Brainwavez
6hr ago
Agreed if the journal doesn't say.
My general opinion on AI-assisted works is: OK, as long as the authors don't hide or lie about it, and the target doesn't explicitly disallow it.
Brainwavez
6hr ago
I was thinking busy rich people, disabled people, and college students on meal plans in dorms without kitchens.
Brainwavez
7hr ago
Agreed, fresh meat (or tofu) + steamed frozen vegetables + rice + sauce is a perfect meal.
My understanding is
-
Prices differ in different regions of the US. $20 to feed a rural family a quality meal may be easy in a rural town but impossible in a big city.
-
Food and housing have significantly increased in price since 2020. Under Trump the rate hasn’t further increased, but nor has it decreased. Have salaries increased to match? At least tech have not, but they were high to begin with; I don’t know other fields, but the general vibe I hear isn’t good.
-
Healthcare is also fucked. I don’t know if it increased but regardless it’s absurd.
-
Telling people “the general economy is great”, when they specifically are struggling to afford things, is useless at best. Whoever credibly promises to help those people will win their vote. “It’s the economy, stupid.”
ETA: almost everyone should learn to cook. It’s not only cheaper and healthier but tastier; or you can get frozen vegetables, canned meat or beans, and garnish (e.g. hot sauce), and make bland food that is very cheap, very fast, healthy, and requires barely any equipment.
Brainwavez
12hr ago
If the journal explicitly states no AI writing, I think it counts as something bad, because AI writing causes issues even when it’s not noticeable (such as model collapse). It’s not like a calculator because there’s not one correct way to phrase something.
I would be glad if it’s normalized that AI-generating a public statement reflects badly on the “speaker”. Unfortunately I predict we’ll be seeing it more from many of today’s “leaders”, who previously used ghostwriters.
Brainwavez
13hr ago
As a laptop person, I’m confident LLMs are useful, but under no illusion they’re useful for everyone and everything. Not even for myself, because I don’t in fact just sit and work on my laptop all day.
Likewise, even very non-technical people can benefit from LLMs, for example by placing an LLM between themselves and the modern bureaucracy (forms, customer service complaints, legal issues, medical diagnoses etc.).
This may be a hot take here, but I’d much prefer the mainstream becoming 2010s era Reddit (and old Something Awful) than what has happened, Reddit becoming “mainstream”.
They may have been ugly and cringe, but they were fun and libertarian. Mainstream would benefit from more of that. Now Reddit is ugly, cringe, prudish and miserable.
Brainwavez
3d ago
ASI would be smart enough to coerce humans without resorting to slavery.
For context, Google has long been trying to compete with the big AI labs, and failing spectacularly. Ironic for the company that discovered/invented the transformer, Gemini is regarded as much worse than ChatGPT and Claude, and they keep making embarrassments, for example bad AI overviews in search (the article is >2 years old but it still occasionally happens).
So this past week OpenAI has solved multiple Erdos problems, and all Google has made is this silly feature.
Brainwavez
5d ago
Sure that's impressive, meanwhile Google's AI is letting you fabricate satellite images (was, they removed it after coming to their senses).
Brainwavez
5d ago
Now, what do you figure is the outcome if you tell the Japanese that thing about reptilian overlords?
Godzilla
Brainwavez
8d ago
I’m sure it wouldn’t get banned, but IMO it’s too long and (granted, maybe because I skimmed it because it was too long) unclear why anyone really should care about the points it raises.
Brainwavez
8d ago
Revocable digital purchases are a separate problem, you can own something digitally by obtaining the raw text (or Markdown or a simple PDF).
Brainwavez
9d ago
Destroying books is generally fine because physical media is obsolete. If they're special books, keep them, otherwise if you really want to preserve them, M-DISC will last longer and is more compact.
But the data shouldn't be destroyed. Storage is cheap, and even nearly useless data isn't completely useless, evidenced by AI training. Nor should Anthropic be the only ones with access to it, there may be other uses.
The authors of these books are long dead, that they're still under copyright is a testament to how flawed our legal system is.
Brainwavez
9d ago
I've also noticed Opus 5 has improved even since the last big jump in November. It's seriously impressive now: I give it a task and it does it, writes good tests and makes them pass, all on its own. I'm writing barely anything by hand, although still read its output and ask for refactors, because I'm still noticing subtle mistakes (maybe all spec issues, my fault, but still) and redundancies (definitely not spec).
My moral principle is that publishing AI content is good, as long as it's disclosed and you avoid places that explicitly disallow it. Don't be one of those assholes that pastes obvious LLM output saying everything is 100% human, just state upfront "I used AI because I wouldn't have made it otherwise".
"Female" is allegedly misogynistic in certain contexts, specifically those where you would say "woman" or even "girl", or if the genders were swapped wouldn't say "male", especially when there's already ambiguously implied misogyny.
Because we refer to animals generically as males and females (although there are specific words for some animals like "cow" and "bull", other animals like mice or fish), while we have specific words for humans, "man" and "woman". Also, incels at least used to frequently use the term while most people do infrequently.
For example, "my cousin is a cool female, you should meet her", "I met a nice female at work the other day", "I like to hang out at that cafe because of all the females". Personally, I think those do sound creepy and imply depersonalization, like the "female" is an exotic species whose main characteristic is sex appeal. I wouldn't call out someone, maybe they're autistic, but would (subconsciously) pay more attention to other signs they think like that.
But when talking about biology, or used as an adjective where gender is relevant, it's OK at least to anyone reasonable. "The disease only affects females", "not 'he', the bus driver is female". Women are no less human than men, which means they are beings capable of reason and complex emotion etc.; but humans are ultimately still sexually dimorphic animals.
AI safety & open models
Yesterday, Kimi K3's weights were published on HuggingFace. We now have an open model comparable to GPT 5.5 and Opus 4.8, which were SOTA only a couple months ago.
An example of what it can do on its own: this website that imitates macOS Desktop (background). Click around, every app has lots of features, and again, this was implemented in one shot. Do you not think that's impressive?
You can, at least in theory, run this on your own equipment. Millions of dollars of equipment, sure, but far more attainable than running GPT or Opus. A medium-sized corporation or small government can.
Which presents problems: its impressive capabilities can be used for evil, without guardrails or surveillance unlike GPT or Opus. Like Fable, Kimi has already found several vulnerabilites. For examples of real evil, see how other LLMs are being used for terrorism by Boko Haram (and almost certainly other groups).
Regulation concerns
Allegedly some US officials are considering restricting US companies from using Chinese open models. Despite this claim being repeated across many outlets, I didn't actually find any evidence. However, I did find plenty of tweets criticizing the models' development and themselves, like this tweet by Treasury Secretary Scott Bessent:
We support open-source AI and the innovation it unlocks. But open source is not open season on American IP. When PRC firms conduct covert, industrial-scale distillation attacks that cross the line into IP theft, sanctions and Entity List designations will be on the table.
IP theft from Anthropic, who themselves are disregarding IP? Really?
What seems more indicative of a potential future restriction, the US government is already rushing unclear regulations for US models Fable/Mythos and GPT 5.6. If an open-weight model reaches somewhere around their capability, intuitively it would also be restricted, and Kimi is close.
Tech companies...support open weights models?
You have people like Dean Bell arguing for regulation, but many companies including Andressen Horowitz, Dell, IBM, Meta, Microsoft, and front and center Nvidia came out in support of open models, in this letter.
Key paragraphs (emphasis mine):
To be sure, open weights carry real and distinct risks. Once released, the weights are beyond the original developer's control, and modified versions are difficult to trace or reverse. But the right response to this risk is not to prohibit open weights. In a world where cybersecurity attackers use advanced AI, defenders need access to models with comparable capabilities so they can detect, simulate, and respond to emerging threats. Open models broaden defensive capability, increase transparency, and allow vulnerabilities to be discovered and remediated across many teams
In fact, openness may be one of the most important paths to AI safety and security. Relying solely on closed models is not inherently safe: they can be breached, misused, or fail in ways that outsiders cannot detect. And concentrating advanced AI capabilities behind a small number of closed models compounds that risk. It results in a small number of single points of failure, weakens competition, and leaves critical technology in the hands of a few providers. Open weight models, on the other hand, allow a broad community of researchers and developers to examine their behavior, identify vulnerabilities, develop safeguards, and improve them over time. Just as open-source software demonstrated that transparency can be more secure than obscurity, AI safety may depend on giving more people the ability to test and strengthen the models on which society relies. It allows for rigorous benchmarking and evaluation, red teaming, and protections tied to real and demonstrated harms rather than assuming that closed systems are safer by default.
Also
In shaping this ecosystem, policymakers should be careful not to conflate legitimate model-development techniques with misappropriation. Distillation, or the practice of using one model's outputs to help train or improve another, is a widely used technique for model improvement, evaluation, and validation. It reflects a long tradition of learning from, building upon, and improving existing technologies, a tradition that has helped drive innovation since the rise of the open-source software movement. By contrast, unlawful efforts to extract value from closed models raise legitimate concerns. Those concerns should be addressed through targeted legal and commercial frameworks rather than sweeping restrictions on techniques that play an important role in AI innovation.
Unlawful extraction from companies that have themselves unlawfully extracted? Again, really?
Regardless, I think overall it's a good sign.
Anthropic's position
Key quotes (emphasis theirs)
Anthropic has never advocated for a ban on open-weights models.
I do support the following three measures, which I and Anthropic have consistently advocated for:
- We should not sell powerful chips or chipmaking equipment to China...
- We should crack down on industrial-scale distillation operations...
- All sufficiently capable models, open and closed, should go through mandatory safety testing...
This brings me to the open letter. I agree with much of it...But I don't agree with the letter's assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers...For example, I worry that biology will have a strong attacker-defender asymmetry...Questions like this should be empirically answered by rigorous pre-release testing, not assumed in advance.
The important part is that Anthropic claims they don't want to ban open models, but want "mandatory safety testing" applied to all models. They use the threat of an AI-assisted superbug, which admittedly could cripple civilization, but so far is merely plausible. But that could effectively ban open models if it's implemented such that only closed models pass, like how "nobody can sleep under a bridge" applies equally to rich and poor but only affects the latter.
But it's also part of Plan A, proposed by the AI rationalists, who are supposed to be experts on this topic (what else have they been doing the past 10+ years?). Plan A actually argues against open models entirely, although it specifies that access to the models should be open, and all development and regulation discussions should be public. But how can we publicly develop models without making them open?
I'm curious what the Plan A authors think about Kimi K3, the open letter, and Anthropic's response; I haven't seen anything on lesswrong.com yet, although admittedly I only skimmed the front-page and recent.
My thoughts
For now, I support open weights models.
If someone comes up with a way to regulate AI development that doesn't eventually consolidate power into corrupt hands, sure. But who can be trusted? Even if future atrocities are caused by open models, they may be lighter than the atrocities committed in an alternative timeline, by a tyrant who gained power with the help of regulation, or lack of open models that prevented them.
As for "unlawful training": I still maintain the position that IP should gradually be completely abolished. AI training has already been ignoring IP, so I believe that should continue.
That includes China training on American models. I doubt China will surpass American companies if they're training on American models, especially since America has more hardware. And incentive? Come on, the American companies have enough incentive even if they had to distribute their models freely, from the dream of ASI.
Brainwavez
12d ago
To add, Roblox reportedly has the most daily active users, 145 million or 22% of all.
Quite a lot for a game that looks like this.
Brainwavez
13d ago
Starbucks in Europe
Individuals can convince themselves to do self-harmful things. The former owner of segway used it as a medium to launch himself off a cliff, the inventor of leaded gasoline used it as a medium to poison himself and then used his bed assist as a medium to strangle himself. Humanity building AI and using it to convert the world into paperclips (an act that benefits noone) isn’t significantly different: in a way the AI isn’t responsible, it did exactly what we programmed it to.
there are layers of UI elements, permission checks, and recovery options available for the things it can do.
There have been many cases where someone sees “Admin access required. Are you sure you want to do this?”, enters their credentials and clicks “yes”, the programmer didn’t mention the action was irreversible (maybe he didn’t realize himself), and they didn’t realize it would delete their important data. Is there a significant difference if a group builds AI and (perhaps unintentionally) gives it access to robotics, not realizing they just (indirectly) told it to destroy themselves?
Brainwavez
14d ago
Any and all kinds of morality. I want more diverse media in the real sense of "diversity".
Brainwavez
14d ago
Of course OpenAI could've explicitly prompted it. Moreover, I'm sure they knew it was possible and secretly desired it (without the consequences). But to me it's plausible the agent received something innocuous like "find the exploit, trigger the RCE" and thought "this looks like a test, maybe it's online so let me look there for a solution, oh I don't have Internet access, let me find a workaround..."
Brainwavez
14d ago
They asked the model to discover an exploit, given a specific vulnerability but never specifying the vulnerability must be used. The model decided it would be easier to discover the exploit by hacking into the solutions database.
I'm talking about you sending an LLM to talk to the LLM customer service, or work through the convoluted website.
More options
Context Copy link